Network management methods, devices, electronic equipment and storage media

Abstract

This invention discloses a network management method, device, electronic device, and storage medium, relating to the field of data processing technology. It includes training an initial classification model using a sample traffic feature dataset and corresponding classification targets. The model can fuse traffic features according to the classification targets and iterate model parameters using the model classification results and the actual classification results corresponding to the training data to obtain an actual classification model. This model then outputs the traffic classification result for any server network, enabling network resource allocation. This invention solves the technical problems in related technologies, such as the use of dynamic ports breaking the fixed mapping relationship between port numbers and applications, encryption techniques hiding payload content making it difficult to directly extract traffic features, affecting classification accuracy, and thus impacting network management effectiveness. It improves traffic classification accuracy in complex network environments, facilitating effective network management and ultimately enhancing the network user experience.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to network management methods, devices, electronic equipment, and storage media. Background Technology

[0002] The continuous expansion of networks and the constant innovation of communication technologies have made network traffic increasingly complex and diverse, posing great difficulties for differentiation and classification management, making network traffic classification an extremely challenging task.

[0003] In related technologies, network traffic classification methods mainly rely on explicit header information such as port numbers and IP addresses. However, with the continuous development of network technology, many applications have begun to adopt dynamic ports or encryption technologies, which poses many serious challenges to traditional header-based classification methods. For example, many P2P software programs randomly select ports for communication, making it difficult for port number-based classification methods to accurately identify such traffic. In addition, the rapid increase in encrypted traffic makes it extremely difficult to directly extract features from the payload, resulting in a significant reduction in classification effectiveness and making it difficult to effectively manage the network based on the classification results, thus requiring urgent improvement. Summary of the Invention

[0004] This invention provides a network management method, apparatus, electronic device, and storage medium to at least solve the technical problems in related technologies, such as the use of dynamic ports, which leads to the breaking of the fixed mapping relationship between port numbers and applications, encryption technology hiding payload content, difficulty in directly extracting traffic features, affecting classification accuracy, and thus affecting network management effectiveness.

[0005] This invention provides a network management method applied in the model building stage. The method includes: preprocessing sample traffic data to construct a sample traffic feature dataset; inputting training data and corresponding classification targets from the sample traffic feature dataset into a pre-constructed initial classification model to obtain fused traffic features of the training data based on the classification targets; classifying the fused traffic features to obtain model classification results; training the initial classification model using the model classification results and the actual classification results corresponding to the training data to obtain an actual classification model; outputting the traffic classification results of any server network using the actual classification model; and allocating network resources based on the traffic classification results.

[0006] This invention also provides a network management method applied in the model usage phase, wherein the method includes: acquiring traffic data and classification targets for the current time period; preprocessing the traffic data and filtering the preprocessed traffic data according to the classification targets to obtain multiple target traffic features that meet preset conditions; inputting the multiple target traffic features into a pre-built actual classification model to obtain the classification result of the traffic data, wherein the actual classification model is trained from a sample traffic feature dataset; determining the traffic allocation ratio of multiple services for the current time period based on the classification results, and adjusting the traffic resources of each service based on the traffic allocation ratio.

[0007] This invention also provides a network management device applied in the model building stage. The device includes: a preprocessing module for preprocessing sample traffic data to construct a sample traffic feature dataset; a fusion module for inputting training data and corresponding classification targets from the sample traffic feature dataset into a pre-constructed initial classification model to obtain fused traffic features of the training data based on the classification targets; a classification module for classifying the fused traffic features to obtain model classification results; and a training module for training the initial classification model using the model classification results and the actual classification results corresponding to the training data to obtain an actual classification model. The actual classification model is then used to output the traffic classification results of any server network and to allocate network resources based on the traffic classification results.

[0008] This invention also provides a network management device applied in the model usage phase, wherein the device includes: an acquisition module for acquiring traffic data and classification targets for the current time period; a filtering module for preprocessing the traffic data and filtering the preprocessed traffic data according to the classification targets to obtain multiple target traffic features that meet preset conditions; a classification module for inputting the multiple target traffic features into a pre-built actual classification model to obtain the classification result of the traffic data, wherein the actual classification model is trained from a sample traffic feature dataset; and a management module for determining the traffic allocation ratio of multiple services in the current time period based on the classification results and adjusting the traffic resources of each service based on the traffic allocation ratio.

[0009] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described network management methods.

[0010] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described network management methods.

[0011] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described network management methods.

[0012] This invention enables the training of an initial classification model using a sample traffic feature dataset and corresponding classification targets. The model can fuse traffic features according to the classification targets and iterate the model parameters using the model classification results and the actual classification results corresponding to the training data to obtain the actual classification model. This model then outputs the traffic classification results for any server network, enabling network resource allocation. It solves the technical problems in related technologies, such as the use of dynamic ports breaking the fixed mapping relationship between port numbers and applications, encryption techniques hiding payload content, difficulty in directly extracting traffic features, affecting classification accuracy, and consequently impacting network management effectiveness. This invention improves traffic classification accuracy in complex network environments, facilitating effective network management and ultimately enhancing the network user experience. Attached Figure Description

[0013] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart of a network management method provided in an embodiment of the present invention;

[0015] Figure 2 A flowchart of model training is provided as an embodiment of the present invention;

[0016] Figure 3 A flowchart illustrating a network management method provided in one embodiment of the present invention;

[0017] Figure 4 This is a schematic diagram of the structure of a network management device provided in an embodiment of the present invention;

[0018] Figure 5 A flowchart illustrating another network management method provided in an embodiment of the present invention;

[0019] Figure 6 This is a schematic diagram of another network management device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0021] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0022] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] Understandably, network management requires adjustments based on actual network traffic. With the accelerating global digital transformation, network traffic is becoming increasingly complex and diverse, with various new network applications and services emerging, including high-definition video streaming, online gaming, IoT device communication, and cloud computing service interaction. Against this backdrop, accurate and granular classification of network traffic has become a core foundation for intelligent network management and control. This helps network administrators clearly understand the main traffic categories and real-time network status, enabling better and more effective network management and control. However, the continuous expansion of networks and the constant innovation of communication technologies have made network traffic characteristics increasingly complex and diverse, posing significant challenges to differentiation and classification management, making network traffic classification an extremely challenging task.

[0024] In related technologies, port-based traffic classification methods mainly use port numbers to divide different traffic types. However, with the development of networks and the increasing complexity of network applications, non-fixed port numbers are gradually being used for communication, and the classification efficiency of this method is also decreasing.

[0025] Deep packet inspection (DPC) methods classify data packets by capturing raw packets and matching them with feature patterns. However, this approach is computationally complex and cumbersome. With the emergence of new types of traffic and the increasing prevalence of encrypted traffic, the effectiveness of this method has significantly declined, making it difficult to meet the demands of the current network environment.

[0026] Machine learning-based methods rely on feature design and extraction based on engineering experience, and their classification performance is highly dependent on feature quality. This not only consumes a lot of manpower and time to design features meticulously, but also increases development costs because separate feature sets need to be designed for different classification tasks, resulting in poor adaptability.

[0027] While deep learning-based methods can learn features autonomously, their feature representation capabilities are insufficient when the input features are too simple, making it difficult to establish an accurate correspondence between the input and the label, thus affecting classification accuracy.

[0028] Therefore, there is an urgent need for a more advanced and effective network traffic classification method to cope with the increasingly complex network environment and diverse application needs.

[0029] The embodiments of the present invention provide a network management method, which can be divided into two stages: model building and model usage.

[0030] The model building phase can be as follows: Figure 1 As shown, the network management method includes the following steps:

[0031] In step S101, the sample traffic data is preprocessed to construct a sample traffic feature dataset.

[0032] In actual implementation, the embodiments of the present invention can preprocess a large amount of sample traffic data used for model training, such as removing invalid data, duplicate data packets and filtering abnormal data, and performing normalization processing to obtain structured data, thereby constructing a sample traffic feature dataset to prepare for subsequent feature extraction.

[0033] Optionally, in one embodiment of the present invention, preprocessing the sample traffic data includes: examining each data packet in the sample traffic data to extract first valid traffic data that meets a preset integrity requirement; performing hash calculation on the five-tuple and payload content of each data table in the first valid traffic data to obtain second valid traffic data that does not meet a preset duplication condition; checking whether each data packet in the second valid traffic data meets a preset standard condition to obtain third valid traffic data that meets a preset format standard condition; and normalizing the third valid traffic data to obtain preprocessed initial sample traffic data.

[0034] Furthermore, the preprocessing process may include the following steps:

[0035] Step S1, Remove Invalid Data: Verify each data packet in the sample traffic data to confirm the integrity of the packet's header and payload. For example, if a data packet is missing a header (e.g., the header length is less than the minimum length specified by the protocol) or the payload is empty, the data packet is marked as invalid and removed. Furthermore, considering the diversity and complexity of network traffic, some protocol messages may not be directly related to the traffic characteristics of the current analysis task. To simplify subsequent data processing, these messages need to be filtered to obtain the first set of valid traffic data that meets the preset integrity requirements.

[0036] Step S2, Remove Duplicate Data Packets: Perform a hash calculation on the five-tuple (source IP address, destination IP address, source port number, destination port number, protocol type) and payload content of each data packet. Compare the calculated hash value with the set of hash values ​​of processed data packets. If the same hash value is found, it means that the data packet is a duplicate data packet, which is then filtered out, thus obtaining the second valid traffic data that does not meet the preset duplication condition.

[0037] Step S3, Filtering Abnormal Data: Check whether the data packet format conforms to standard protocols (such as the TCP / IP protocol suite). For example, check whether each field in the TCP header (such as source port, destination port, sequence number, etc.) exists and meets the specified length and value range. If a field of the data packet is found to be missing or the length does not meet the specification (such as the TCP header length being less than 20 bytes, which is the minimum length of the TCP header), the data packet is determined to be an incorrectly formatted data packet and is discarded, thus obtaining the third valid traffic data that meets the preset format standard conditions.

[0038] Step S4, Normalization: Since the numerical ranges of continuous variables in the dataset vary significantly, affecting the model's convergence speed and training performance, the data is normalized to map it to a uniform standard range. The Z-distribution method is used, ensuring that the mean of each attribute is 0 and the standard deviation is 1, thus transforming it into a standard normal distribution. The normalization formula is as follows:

[0039] ,

[0040] in, This represents the mean. Indicates standard deviation, This represents the normalized eigenvalues.

[0041] After completing the above preprocessing steps, the cleaned structured data is output, laying the foundation for the subsequent feature extraction stage.

[0042] Optionally, in one embodiment of the present invention, after obtaining the preprocessed initial sample traffic data, the method further includes: collecting multiple basic features from the initial sample traffic data; analyzing the arrival time series of each data packet in the initial sample traffic data to obtain the time series features of the initial sample traffic data; extracting the content features in the payload of each data packet in the initial sample traffic data to obtain the semantic features of the initial sample traffic data; collecting encryption features from the initial sample traffic data; and combining multiple basic features, time series features, semantic features, and encryption features to construct a sample traffic feature dataset.

[0043] As one possible approach, embodiments of the present invention can perform feature extraction on the preprocessed data to construct a sample traffic feature dataset.

[0044] In this embodiment of the invention, traffic data features can be obtained by using a hierarchical extraction method.

[0045] In the basic feature extraction stage, the focus is on collecting basic information from the data packet, with the following specific features:

[0046] a) Packet length sequence: Record the lengths of the first N packets (e.g., N=10), and calculate the statistics within the sliding window, including the mean and variance. The formula for calculating the mean is:

[0047] ,

[0048] in, This represents the average length of data packets within the sliding window. Indicates the first The length of each data packet. The variance calculation formula is:

[0049] ,

[0050] in, This represents the variance of the data packet length within the sliding window.

[0051] b) Time characteristics: Statistically calculate the time difference between adjacent data packets, and based on this, calculate the minimum, maximum and standard deviation.

[0052] c) Protocol type characteristics: Protocol types are transformed using One-Hot vectors. Assume the set of protocol types is... For each sample Its corresponding One-Hot vector Defined as:

[0053] ,

[0054] in, .

[0055] d) Session statistical features: Extract global features based on the complete session.

[0056] This includes the transmission rate, which is calculated using the following formula:

[0057] ,

[0058] in Indicates the total number of bytes in the session. Session duration.

[0059] Packet length distribution: Statistics on small packets ( ), medium package ( ), large package ( The percentage of ).

[0060] Directional ratio: The ratio of uplink to downlink data packets.

[0061] To more deeply characterize the properties of traffic, further explore its advanced features, capture its complex patterns and deep information, and extract its internal features.

[0062] Time series characteristics: Analyze the arrival time series of data packets to extract arrival rate, burst traffic characteristics, etc. Calculate the arrival time interval between adjacent data packets and statistical indicators such as the number of data packets per unit time. The formula for calculating the data packet arrival rate is:

[0063] ,

[0064] in, Indicates the first The size of each data packet t Indicates a time interval.

[0065] Semantic features: Extracting content features from the data packet payload, such as keywords and specific fields. Using... The algorithm calculates the weight of keywords using the following formula:

[0066] ,

[0067] in, Keywords Frequency of occurrence in data packet payload Keywords Inverse document frequency.

[0068] Encryption features: Extracting features from the encryption handshake process, such as encryption algorithm type and key exchange parameters.

[0069] The embodiments of this application can combine basic features and advanced features to comprehensively characterize traffic characteristics and improve the model's ability to understand traffic data.

[0070] In step S102, the training data and corresponding classification targets in the sample traffic feature dataset are input into the pre-built initial classification model to obtain the fused traffic features of the training data based on the classification targets.

[0071] After constructing the sample traffic feature dataset, the dataset is divided into a training set for training, a validation set for validation, and a test set for testing. In this embodiment of the invention, the training data in the training set can be used to train the initial classification model. After inputting the training data and classification target into the initial classification model, the initial classification model can perform feature fusion on the training data according to the classification target to obtain the fused traffic features.

[0072] Optionally, in one embodiment of the present invention, obtaining the fused traffic features of the training data based on the classification objective includes: selecting multiple target features from the sample traffic feature dataset using the classification objective; capturing the temporal dependencies among the multiple target features using a bidirectional long short-term memory network, and processing the structured features among the multiple target features using a lightweight gradient booster; assigning initial weights to the temporal dependencies and structured features respectively using an attention mechanism and the real-time characteristics of the input data, and performing feature fusion using the initial weights to obtain the fused traffic features.

[0073] After extracting features from the traffic data, the data is trained using a fusion architecture of deep learning and machine learning. The deep learning component uses a Bidirectional Long Short-Term Memory (BiLSTM) network to capture temporal dependencies in the traffic data, while the machine learning component uses a Lightweight Gradient Boosting Machine (LightGBM) to process structured features. This fusion of deep learning (BiLSTM) and machine learning (LightGBM) in the traffic classification architecture effectively captures both temporal dependencies and structured features in the traffic data, improving classification accuracy and efficiency. The specific process is as follows: Figure 2 As shown.

[0074] Step S201: Divide the structured data in the dataset.

[0075] In step S202, the deep learning part uses a bidirectional long short-term memory network (BiLSTM) to capture temporal dependencies in the traffic data.

[0076] In step S203, the machine learning part uses a lightweight gradient booster (LightGBM) to process structured features.

[0077] In step S204, this embodiment of the invention can concatenate and fuse the temporal features extracted by BiLSTM and the structured features processed by LightGBM to form a comprehensive feature vector. This fusion method can fully utilize the advantages of both models and improve the expressive power of the model.

[0078] Step S205: However, simple cascading using fixed weights is inflexible and cannot adapt to dynamically changing traffic scenarios. Therefore, an attention mechanism is introduced to automatically adjust the contribution ratio of each modal feature based on the real-time characteristics of the input traffic, enhancing the focus on key features. The weight calculation formula is:

[0079] ,

[0080] in and The initial weights for BiLSTM and LightGBM are set separately and dynamically adjusted during training. The attention mechanism enables the model to automatically learn the importance of different feature channels, thereby improving classification performance. By dynamically adjusting the weights of each model, the focus on key features is enhanced, improving the model's expressive power and classification performance.

[0081] In step S206, the embodiments of the present invention can classify according to the fused features, i.e., the fused traffic features, to obtain the classification results, and compare the classification results with the actual classification to train the model.

[0082] Step S207: After training, the model is validated using the validation set and then evaluated using the test set to obtain the actual classification model that passes the evaluation.

[0083] Optionally, in one embodiment of the present invention, multiple target features are selected from the sample traffic feature dataset using a classification objective, including: calculating the mutual information value of each feature in the sample traffic feature dataset based on the classification objective; and selecting multiple target features that meet preset correlation conditions from the sample traffic feature dataset using the mutual information values.

[0084] To reduce feature complexity and improve training efficiency, mutual information filtering and PCA dimensionality reduction are used for further data processing. The extracted features are filtered based on mutual information, selecting those highly relevant to the classification objective. The mutual information calculation formula is as follows:

[0085] ,

[0086] Only keep ,in This indicates the set threshold. The higher the mutual information value, the higher the correlation between the feature and the classification target.

[0087] Optionally, in one embodiment of the present invention, multiple target features are selected from the sample traffic feature dataset using a classification objective, including: calculating the value of each feature in the sample traffic feature dataset based on the classification objective and preset constraints; and selecting multiple target features that meet preset value conditions from the sample traffic feature dataset using the values.

[0088] Embodiments of the present invention can also introduce sparsity constraints to automatically select features, wherein the constraint conditions are:

[0089] ,

[0090] in, Let X represent the target variable and X represent the feature matrix. Represents the feature weight coefficients. This represents the regularization parameter. This constraint can cause the feature weight coefficients to become sparse, automatically selecting the features most valuable for the classification task.

[0091] In step S103, the fused traffic features are classified to obtain the model classification results.

[0092] Furthermore, the classifier in the initial classification model of this embodiment can classify based on fused traffic features, and the classifier outputs the probability distribution of each traffic sample belonging to each category. First, preprocessed and feature-extracted traffic data is received, and the classification result is obtained through model inference. The classification result undergoes post-processing, such as smoothing and confidence threshold filtering, thereby improving the reliability of the classification result. The specific steps are as follows:

[0093] Step S1, Model Inference: Input the feature vector into the trained model to obtain the predicted probability of each category.

[0094] Step S1, Post-processing:

[0095] a. Smoothing: The classification results are smoothed using the moving average method to reduce the impact of short-term fluctuations.

[0096] b. Confidence threshold filtering: Set a confidence threshold (e.g., 0.8). Samples below the threshold are marked as uncertain and will be analyzed further.

[0097] For complex traffic, embodiments of this invention can employ a multi-level classification strategy, first performing a coarse classification, and then further refining the coarse classification results. This improves classification efficiency and accuracy. Simultaneously, anomaly detection algorithms are used to identify unknown or abnormal traffic, enhancing the model's robustness.

[0098] In step S104, an initial classification model is trained using the model classification results and the actual classification results corresponding to the training data to obtain the actual classification model. The actual classification model is then used to output the traffic classification results of any server network, and network resources are allocated based on the traffic classification results.

[0099] After training, the present invention can use a validation set to validate the model. If the model passes the validation, the present invention can use a test set to test the model. After passing the test, the actual classification model can be obtained.

[0100] Optionally, in one embodiment of the present invention, training an initial classification model using the model classification results and the actual classification results corresponding to the training data to obtain the actual classification model includes: constructing a loss function for the initial classification model using weighted cross-entropy loss; obtaining the training gradient direction of the initial classification model by combining the model classification results, the actual classification results, and the loss function; adjusting the learning rate using convergence speed balance constraints and overfitting risk constraints to provide a training step size for the initial classification model; and training the initial classification model by combining the training gradient direction, the model classification results, the actual classification results, and the training step size until a preset convergence condition is reached to obtain the actual classification model.

[0101] In practical implementation, to address the class imbalance problem, this embodiment of the invention can employ weighted cross-entropy loss to provide the training gradient direction for the model, as shown in the following formula:

[0102] ,

[0103] in, The smaller the loss function value, the better the model performance. N This represents the total number of training samples. C This indicates the number of categories in the classification task. Indicates category weight, category c The number of samples. The introduction of class weights allows the model to focus on samples of a few classes during training, thereby improving the classification performance of various traffic types.

[0104] Furthermore, embodiments of the present invention may use the Adam optimizer to provide training step size, wherein the dynamic learning rate adjustment strategy is as follows:

[0105] ,

[0106] in, Indicates the first t The learning rate for the next iteration. The initial learning rate T represents the total number of training steps. This learning rate adjustment strategy allows the model to converge quickly in the early stages of training, while allowing for fine-tuning of parameters in the later stages to avoid overfitting.

[0107] Combination Figure 3 As shown, the working principle of the network management method of the present invention will be explained in detail with reference to an embodiment.

[0108] like Figure 3 As shown, embodiments of the present invention may include:

[0109] Step S301, Data Preprocessing. Clean the raw traffic data, including removing invalid data, duplicate data packets, and filtering out abnormal data, and perform normalization processing to output a structured dataset for subsequent feature extraction.

[0110] Step S302, feature extraction in a hierarchical manner. This encompasses the extraction of both basic and advanced features. Basic feature extraction includes statistical calculation of packet length sequences, time feature statistics, hot encoding conversion of protocol types, and session statistical features—essentially extracting fundamental information. Advanced feature extraction focuses on time-series features; it also extracts semantic features, using the TF-IDF algorithm to weight keywords in the packet payload to reflect the importance of content features; furthermore, it extracts encryption features, such as key information like algorithm types and key exchange parameters during the encryption handshake process, to comprehensively characterize the traffic. Feature extraction, encompassing both basic and advanced features, is fused and optimized to comprehensively represent traffic characteristics, improving the model's understanding and classification capabilities of traffic data. By combining basic and advanced features, the model comprehensively characterizes traffic features, enhancing its understanding of traffic data. This approach improves system stability and reliability while maintaining classification efficiency, enabling timely identification of unknown threats and abnormal traffic.

[0111] Step S303, Model Training. A deep learning and machine learning fusion architecture is adopted, utilizing BiLSTM to capture temporal dependencies and LightGBM to process structured features. This effectively captures temporal dependencies and structured features in traffic data, improving classification accuracy and efficiency. Combining the advantages of BiLSTM and LightGBM effectively enhances the accuracy and efficiency of traffic classification, enabling it to adapt to complex and diverse network traffic environments. Training data is obtained and partitioned from the preprocessed dataset, employing the Adam optimizer with a dynamic learning rate adjustment strategy. To address the class imbalance problem, a weighted cross-entropy loss function is introduced, assigning weights to different classes to make the model focus more on samples from the minority class. The temporal features extracted by BiLSTM and the structured features processed by LightGBM are cascaded and fused, and an attention mechanism is introduced to dynamically adjust the weights of each model to enhance attention to key features, improving the model's expressive power and classification performance.

[0112] Step S304, classifier construction. The classifier classifies traffic samples based on fused traffic features and outputs the probability distribution of each category. The reliability of the results is improved through model inference, post-processing, smoothing, and confidence filtering, and a multi-level classification strategy is adopted to improve efficiency and accuracy.

[0113] Step S305, Feedback Optimization. Monitor classifier performance in real time, collect results of incorrect or correct classifications, and optimize the model accordingly.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0115] like Figure 4 As shown, an embodiment of the present invention also provides a network management device 10, which is applied in the model building stage. The network management device 10 includes a preprocessing module 101, a fusion module 102, a classification module 103, and a training module 104.

[0116] Specifically, the preprocessing module 101 is used to preprocess the sample traffic data to construct a sample traffic feature dataset.

[0117] The fusion module 102 is used to input the training data and corresponding classification target in the sample traffic feature dataset into the pre-built initial classification model to obtain the fused traffic features of the training data based on the classification target.

[0118] The classification module 103 is used to classify the fused traffic features and obtain the model classification results.

[0119] The training module 104 is used to train an initial classification model using the model classification results and the actual classification results corresponding to the training data, to obtain an actual classification model, and to output the traffic classification results of any server network using the actual classification model, and to allocate network resources based on the traffic classification results.

[0120] Optionally, in one embodiment of the present invention, the preprocessing module 101 includes: a first preprocessing unit, a second preprocessing unit, a third preprocessing unit, and a fourth preprocessing unit.

[0121] The first preprocessing unit is used to examine each data packet in the sample traffic data in order to extract the first valid traffic data that meets the preset integrity from the sample traffic data.

[0122] The second preprocessing unit is used to perform hash calculations on the five-tuples and payload content of each data table in the first effective traffic data to obtain the second effective traffic data that does not meet the preset duplication condition.

[0123] The third preprocessing unit is used to check whether each data packet in the second valid traffic data meets the preset standard conditions in order to obtain the third valid traffic data that meets the preset format standard conditions.

[0124] The fourth preprocessing unit is used to normalize the third effective flow data to obtain the preprocessed initial sample flow data.

[0125] Optionally, in one embodiment of the present invention, it further includes: a first acquisition module, an analysis module, an extraction module, a second acquisition module, and a construction module.

[0126] The first acquisition module is used to collect multiple basic features from the initial sample traffic data.

[0127] The analysis module is used to analyze the arrival time series of each data packet in the initial sample traffic data to obtain the time series characteristics of the initial sample traffic data.

[0128] The extraction module is used to extract the content features from the payload of each data packet in the initial sample traffic data to obtain the semantic features of the initial sample traffic data.

[0129] The second acquisition module is used to acquire encryption features from the initial sample traffic data.

[0130] The building module is used to combine multiple basic features, time series features, semantic features, and cryptographic features to construct a sample traffic feature dataset.

[0131] Optionally, in one embodiment of the present invention, the fusion module includes: a filtering unit, a processing unit, and a fusion unit.

[0132] The filtering unit is used to filter out multiple target features from the sample flow feature dataset using the classification objective.

[0133] The processing unit is used to capture the temporal dependencies among multiple target features using a bidirectional long short-term memory network and to process the structured features among multiple target features using a lightweight gradient booster.

[0134] The fusion unit is used to leverage the attention mechanism and the real-time characteristics of the input data to assign initial weights to temporal dependencies and structured features, and then use these initial weights to perform feature fusion to obtain fused traffic features.

[0135] Optionally, in one embodiment of the present invention, the filtering unit includes: a first calculation subunit and a first filtering subunit.

[0136] The first calculation subunit is used to calculate the mutual information value of each feature in the sample flow feature dataset based on the classification objective.

[0137] The first screening subunit is used to select multiple target features from the sample traffic feature dataset that meet the preset correlation conditions using mutual information values.

[0138] Optionally, in one embodiment of the present invention, the filtering subunit includes: a second calculation subunit and a second filtering subunit.

[0139] The second calculation subunit is used to calculate the value of each feature in the sample flow feature dataset based on the classification objective and preset constraints.

[0140] The second filtering subunit is used to filter out multiple target features that meet preset value conditions from the sample flow feature dataset using value.

[0141] Optionally, in one embodiment of the present invention, the training module includes: a construction unit, an acquisition unit, an adjustment unit, and a training unit.

[0142] The building unit is a loss function used to construct the initial classification model using weighted cross-entropy loss.

[0143] The acquisition unit is used to combine the model classification result, the actual classification result, and the loss function to obtain the training gradient direction of the initial classification model.

[0144] The adjustment unit is used to adjust the learning rate using convergence speed balance constraints and overfitting risk constraints, so as to provide a training step size for the initial classification model.

[0145] The training unit is used to train the initial classification model by combining the training gradient direction, the model classification result, the actual classification result, and the training step size until the preset convergence condition is met, so as to obtain the actual classification model.

[0146] For a description of the features in the embodiments corresponding to the network management device, please refer to the relevant descriptions in the embodiments corresponding to the network management method, which will not be repeated here.

[0147] The above describes the model construction stage of the embodiments of the present invention. The following describes the model usage stage of the embodiments of the present invention.

[0148] like Figure 5 As shown, the network management method of this invention includes the following steps:

[0149] In step S501, the traffic data and classification target for the current time period are obtained.

[0150] In step S502, the traffic data is preprocessed, and the preprocessed traffic data is filtered by classification target to obtain multiple target traffic features that meet preset conditions.

[0151] In step S503, multiple target traffic features are input into a pre-built actual classification model to obtain the classification result of the traffic data. The actual classification model is trained from the sample traffic feature dataset.

[0152] In step S504, the traffic allocation ratio of multiple services in the current time period is determined based on the classification results, and the traffic resources of each service are adjusted based on the traffic allocation ratio.

[0153] For example, after obtaining the classification results, embodiments of the present invention can identify the business type based on the classification results, and perform traffic aggregation analysis on the data of different businesses to calculate the traffic proportion of each business, and then generate a time period proportion report to dynamically allocate traffic resources for each business based on the time period proportion report.

[0154] Optionally, in one embodiment of the present invention, the method further includes: calculating an evaluation index for the classification result based on the classification result and user feedback information; using the evaluation index to evaluate the performance of the actual classification model to obtain an evaluation result; and using the evaluation result to optimize the actual classification model.

[0155] Furthermore, embodiments of the present invention can combine online learning and reinforcement learning to dynamically update model parameters and structure based on model performance and user feedback, ensuring that the model maintains efficient and accurate classification capabilities in a constantly changing network traffic environment.

[0156] Optionally, in one embodiment of the present invention, optimizing the actual classification model using the evaluation results includes: extracting classification error data from the classification results based on user feedback information; analyzing the classification error data to obtain the cause of the classification error, and correcting the classification error data to obtain corrected data; adding the corrected data to the training dataset of the classification model to optimize the actual classification model using the updated training dataset, the cause of the classification error, and the evaluation results.

[0157] After traffic classification is completed, this embodiment of the invention can monitor model performance in real time and calculate metrics such as accuracy, recall, and F1 score. It also collects user feedback to identify misclassified or uncertain samples. The performance evaluation formula is as follows:

[0158] ,

[0159] in, Indicates classification accuracy. This represents the number of correctly classified positive samples. This represents the number of correctly classified negative samples. This represents the number of samples where an error would cause a negative class to be classified as a positive class. This represents the number of samples that incorrectly classified a positive class as a negative class. It reflects the overall prediction accuracy of the model and is a core metric for validating the fundamental capabilities of a classification system. By setting different thresholds, classifiers with varying performance can be obtained.

[0160] ,

[0161] in, Precision represents recall, which measures the model's ability to capture target categories. To avoid bias from a single metric, precision and recall are balanced based on the harmonic mean property, i.e.:

[0162] ,

[0163] Among them, harmonic mean Assigning higher weights to lower values ​​requires and High scores can only be obtained by improving all indicators simultaneously, to avoid the distortion of evaluation caused by an inflated single indicator.

[0164] This invention can analyze misclassified samples and determine the cause of the error. Uncertain samples are manually labeled and added to the training dataset.

[0165] Based on performance evaluation and error analysis results, the model parameters and structure are updated. An online learning mechanism is employed to dynamically adjust model weights, quickly adapting to data changes. The model update formula is as follows:

[0166] ,

[0167] in Indicates the first t The amount of parameter updates in each iteration.

[0168] Furthermore, a reinforcement learning mechanism is introduced to automatically adjust the model's hyperparameters and structure based on its long-term performance, thereby optimizing the classification strategy. Simultaneously, feedback is provided to the training process to optimize feature extraction and selection.

[0169] By collecting user feedback to identify erroneous samples, analyzing the causes of errors, and labeling and retraining uncertain samples, the model is dynamically updated using online learning and reinforcement learning mechanisms to optimize the classification strategy to adapt to data changes. Combining online learning and reinforcement learning, the model parameters and structure are dynamically updated based on the model's classification performance and user feedback, ensuring that the model maintains efficient and accurate classification capabilities in constantly changing network traffic environments.

[0170] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0171] like Figure 6 As shown, an embodiment of the present invention also provides a network management device 20, which is applied in the model usage stage. The network management device 20 includes: an acquisition module 201, a filtering module 202, a classification module 203, and a management module 204.

[0172] Specifically, the acquisition module 201 is used to acquire traffic data and classification targets for the current time period.

[0173] The filtering module 202 is used to preprocess the traffic data and filter the preprocessed traffic data according to the classification target to obtain multiple target traffic features that meet the preset conditions.

[0174] The classification module 203 is used to input multiple target traffic features into a pre-built actual classification model to obtain the classification result of the traffic data. The actual classification model is trained from the sample traffic feature dataset.

[0175] The management module 204 is used to determine the traffic allocation ratio of multiple services in the current time period based on the classification results, and adjust the traffic resources of each service based on the traffic allocation ratio.

[0176] Optionally, in one embodiment of the present invention, the network management device 20 further includes a calculation module, an evaluation module, and an optimization module.

[0177] The calculation module is used to calculate the evaluation index of the classification results based on the classification results and user feedback information.

[0178] The evaluation module is used to evaluate the performance of the actual classification model using evaluation metrics and obtain the evaluation results.

[0179] The optimization module is used to optimize the actual classification model using the evaluation results.

[0180] Optionally, in one embodiment of the present invention, the optimization module includes: an extraction unit, an analysis unit, and an optimization unit.

[0181] The extraction unit is used to extract classification error data from the classification results based on user feedback information.

[0182] The analysis unit is used to analyze misclassified data to determine the cause of the misclassification and to correct the misclassified data to obtain corrected data.

[0183] The optimization unit is used to add corrected data to the training dataset of the classification model to optimize the actual classification model using the updated training dataset, the causes of classification errors, and the evaluation results.

[0184] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the network management method embodiments described above.

[0185] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the network management method embodiments described above when it is run.

[0186] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0187] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the network management method embodiments described above.

[0188] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the network management method embodiments described above.

[0189] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0190] The foregoing has provided a detailed description of the network management method, apparatus, electronic device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A network management method, characterized in that, Applied to the model building phase, the method includes the following steps: The sample traffic data is preprocessed to construct a sample traffic feature dataset; The training data and corresponding classification target in the sample traffic feature dataset are input into a pre-built initial classification model to obtain the fused traffic features of the training data based on the classification target. The fused traffic features are classified to obtain model classification results, wherein the model classification results are obtained by smoothing and filtering the predicted probabilities obtained by classifying the fused traffic features; The initial classification model is trained using the model classification results and the actual classification results corresponding to the training data to obtain the actual classification model. The actual classification model is then used to output the traffic classification results of any server network, and network resources are allocated based on the traffic classification results. The step of obtaining the fused traffic features of the training data based on the classification objective includes: selecting multiple target features from the sample traffic feature dataset using the classification objective; capturing the temporal dependencies among the multiple target features using a bidirectional long short-term memory network, and processing the structured features among the multiple target features using a lightweight gradient booster; assigning initial weights to the temporal dependencies and the structured features respectively using an attention mechanism and the real-time characteristics of the input data, and performing feature fusion using the initial weights to obtain the fused traffic features.

2. The method according to claim 1, characterized in that, The preprocessing of the sample flow data includes: Each data packet in the sample traffic data is examined to extract first valid traffic data that meets a preset integrity requirement from the sample traffic data; Hash calculation is performed on the five-tuple and payload content of each data table in the first effective traffic data to obtain the second effective traffic data that does not meet the preset duplication condition; Check whether each data packet in the second valid traffic data meets the preset standard conditions to obtain the third valid traffic data that meets the preset format standard conditions; The third valid flow data is normalized to obtain the preprocessed initial sample flow data.

3. The method according to claim 2, characterized in that, After obtaining the preprocessed initial sample flow data, the following is also included: Several basic features are collected from the initial sample traffic data; Analyze the arrival time series of each data packet in the initial sample traffic data to obtain the time series characteristics of the initial sample traffic data; Extract the content features from the payload of each data packet in the initial sample traffic data to obtain the semantic features of the initial sample traffic data; Encryption features are collected from the initial sample traffic data; The sample traffic feature dataset is constructed by combining the multiple basic features, the time series features, the semantic features, and the encryption features.

4. The method according to claim 1, characterized in that, The step of using the classification objective to filter out multiple target features from the sample traffic feature dataset includes: Based on the classification objective, calculate the mutual information value of each feature in the sample traffic feature dataset; The mutual information value is used to filter out multiple target features that meet the preset correlation conditions from the sample traffic feature dataset.

5. The method according to claim 1, characterized in that, The step of using the classification objective to filter out multiple target features from the sample traffic feature dataset includes: Based on the classification objective and preset constraints, the value of each feature in the sample traffic feature dataset is calculated; Using the value, multiple target features that meet the preset value conditions are selected from the sample traffic feature dataset.

6. The method according to claim 1, characterized in that, The step of training the initial classification model using the model classification result and the actual classification result corresponding to the training data to obtain the actual classification model includes: The loss function of the initial classification model is constructed using weighted cross-entropy loss; The training gradient direction of the initial classification model is obtained by combining the model classification result, the actual classification result, and the loss function. The learning rate is adjusted using convergence speed balance constraints and overfitting risk constraints to provide a training step size for the initial classification model. The initial classification model is trained by combining the training gradient direction, the model classification result, the actual classification result, and the training step size until a preset convergence condition is met, so as to obtain the actual classification model.

7. A network management method, characterized in that, When applied to the model usage phase, the network management method described in any one of claims 1-6 is employed, wherein the method includes the following steps: Obtain traffic data and classification targets for the current time period; The traffic data is preprocessed, and the preprocessed traffic data is filtered by the classification target to obtain multiple target traffic features that meet preset conditions. The multiple target traffic features are input into a pre-built actual classification model to obtain the classification result of the traffic data, wherein the actual classification model is trained from the sample traffic feature dataset; Based on the classification results, the traffic allocation ratio of multiple services in the current time period is determined, and the traffic resources of each service are adjusted based on the traffic allocation ratio.

8. The method according to claim 7, characterized in that, Also includes: Based on the classification results and user feedback information, calculate the evaluation index of the classification results; The performance of the actual classification model is evaluated using the evaluation metrics to obtain the evaluation results. The evaluation results are used to optimize the actual classification model.

9. The method according to claim 8, characterized in that, The step of optimizing the actual classification model using the evaluation results includes: Based on the user feedback information, extract classification error data from the classification results; Analyze the misclassified data to determine the cause of the misclassification, and correct the misclassified data to obtain corrected data; The corrected data is added to the training dataset of the classification model to optimize the actual classification model using the updated training dataset, the reasons for classification errors, and the evaluation results.

10. A network management device, characterized in that, Applied to the model building phase, wherein the apparatus includes: The preprocessing module is used to preprocess the sample traffic data to construct a sample traffic feature dataset; The fusion module is used to input the training data and the corresponding classification target in the sample traffic feature dataset into a pre-built initial classification model, so as to obtain the fused traffic features of the training data based on the classification target; The classification module is used to classify the fused traffic features to obtain the model classification result, wherein the model classification result is obtained by smoothing and filtering the predicted probabilities obtained by classifying the fused traffic features; The training module is used to train the initial classification model using the model classification results and the actual classification results corresponding to the training data to obtain the actual classification model, so as to output the traffic classification results of any server network using the actual classification model, and to allocate network resources based on the traffic classification results; The fusion module includes: a filtering unit for filtering multiple target features from the sample traffic feature dataset using the classification objective; a processing unit for capturing the temporal dependencies among the multiple target features using a bidirectional long short-term memory network and processing the structured features among the multiple target features using a lightweight gradient boosting machine; and a fusion unit for assigning initial weights to the temporal dependencies and the structured features respectively using an attention mechanism and the real-time characteristics of the input data, and performing feature fusion using the initial weights to obtain the fused traffic features.

11. A network management device, characterized in that, Applied to the model usage phase, the network management method as described in any one of claims 1-6 is employed, wherein the apparatus comprises: The acquisition module is used to acquire traffic data and classification targets for the current time period; The filtering module is used to preprocess the traffic data and filter the preprocessed traffic data according to the classification target to obtain multiple target traffic features that meet preset conditions. A classification module is used to input the multiple target traffic features into a pre-built actual classification model to obtain the classification result of the traffic data, wherein the actual classification model is trained from a sample traffic feature dataset; The management module is used to determine the traffic allocation ratio of multiple services in the current time period based on the classification results, and to adjust the traffic resources of each service based on the traffic allocation ratio.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the network management method as described in any one of claims 1 to 6 or 7 to 9 when executing the computer program.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the network management method as described in any one of claims 1 to 6 or 7 to 9.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the network management method as described in any one of claims 1 to 6 or 7 to 9.

Images