Federal network traffic classification method and system facing data and computing power dual heterogeneity

Abstract

The application belongs to the technical field of network traffic classification, and discloses a federal network traffic classification method and system facing data and computing power double heterogeneity. Instruction fine-tuning data set is constructed based on network traffic data of each client node, and local model pruning rate is determined according to available computing resources of the node; the same structured pruning network traffic classification model is allocated to nodes with the same pruning rate; local iterative training is carried out by using low-rank fine-tuning and client-group correction mechanism to obtain structured pruning network traffic classification model parameter increment; the conflict increment is projected and corrected and the norm is scaled by using the previous round of global model parameter increment direction, and then the parameter mapping and aggregation according to the channel alignment are carried out to obtain the global model parameter increment, and the model is iteratively updated until convergence. The network traffic data heterogeneity and client computing power heterogeneity are cooperatively adapted, the training burden of weak resource nodes is reduced, the model drift and gradient conflict are relieved, and the model convergence stability and traffic classification performance are improved.

Description

Technical Field

[0001] This invention relates to the field of network traffic classification technology, and in particular to a method and system for classifying federated network traffic in cases of dual heterogeneity in data and computing power. Background Technology

[0002] With the rapid development of internet technology and the continuous expansion of network scale, network traffic classification, as a fundamental link in network management, security monitoring, and service quality assurance, has become increasingly important. In recent years, data-driven methods based on machine learning and deep learning have been introduced into this field, classifying network traffic by extracting statistical or temporal features. However, traditional deep learning models typically rely on centralized data collection and training, which faces serious data silo problems and increasingly stringent user privacy protection issues in practical applications. To address the privacy leaks and data silo dilemmas caused by centralized training, federated learning has been introduced into the field of network traffic classification. This paradigm not only effectively protects the privacy and security of network data but also fully utilizes the massive computing resources distributed at the network edge and multi-source heterogeneous traffic data, greatly improving the generalization ability and security of traffic classification systems.

[0003] As cyberattacks become more covert and application layer protocols become more complex, traditional lightweight federated learning models are showing their limitations in representing features when dealing with highly nonlinear, dynamically changing, and complex network traffic (such as cross-domain encrypted traffic and zero-day attack traffic). Therefore, the industry has begun to combine large language models with strong feature extraction and context understanding capabilities with federated learning, using federated learning-based large models to mine data features more deeply, thereby improving the accuracy of network traffic classification in complex network environments.

[0004] However, real-world network environments consist of a massive number of heterogeneous nodes. Different nodes (such as high-performance edge servers and low-power IoT gateways) vary significantly in computing power, storage capacity, and communication bandwidth. Existing federated learning-based large language models often assume that all participating nodes have equal or similar resource conditions, forcing all nodes to complete model training of the same scale and complexity. This one-size-fits-all approach leads to the following problems: 1. Semantic space incompatibility and gradient conflicts. Specifically, different clients retain different model feature channels based on their computing power (e.g., client A retains channels numbered 1, 3, and 5, while client B retains channels numbered 1, 2, and 6). This results in gradient information generated by clients during local training being in different subspaces. This structural difference causes severe semantic space incompatibility. Even when multiple clients face the same network traffic classification task, their updated gradients will generate severe gradient conflicts during aggregation. Simple weighted aggregation methods struggle to handle this dimensionality mismatch and semantic shift, leading to difficulties in global model convergence and affecting classification. 1. Accuracy; 2. In real network environments, network traffic in different geographical locations or business scenarios exhibits significant statistical heterogeneity. When heterogeneous model structures are combined with non-independently distributed traffic data, the local evolution direction of each client will undergo verification shift. This not only makes it difficult for the local optimal solution (the heterogeneous model structure of a single client) to approach the global optimal solution (the entire classification model), but also further exacerbates the difficulty of aggregation between heterogeneous models, resulting in a significant decrease in the classification model's recognition accuracy for rare protocols or bursty traffic; 3. Under the existing federated learning framework, the performance loss caused by data heterogeneity and the performance loss caused by model structure heterogeneity (introduced by pruning) are often intertwined. The negative damage to the system caused by data heterogeneity and model heterogeneity is highly coupled. Currently, there is a lack of effective technical means to accurately decouple these two different sources of interference. Since it is impossible to distinguish whether the performance degradation is caused by uneven traffic distribution or by structural conflicts after pruning, targeted optimization strategies (such as gradient mapping or knowledge distillation) are difficult to achieve the expected results in heterogeneous environments.

[0005] In summary, existing network traffic classification methods based on federated learning large models lack adaptability to the computing resources of heterogeneous devices in real network environments. This leads to devices with weaker computing power being unable to participate in the training of the network traffic classification model due to resource overload, or causes a severe "fallback effect," affecting the convergence speed of the classification model and the accuracy of network traffic classification. Summary of the Invention

[0006] Therefore, the technical problem to be solved by the present invention is to overcome the lack of adaptability of existing network traffic classification methods based on federated learning large models to the computing resources of heterogeneous devices in real network environments. This results in devices with weak computing power being unable to participate in the training of the network traffic classification model due to resource overload, or causing a serious fall-out effect, which affects the convergence speed of the classification model and the accuracy of network traffic classification.

[0007] To address the aforementioned technical problems, this invention provides a method for classifying federated network traffic in cases of dual heterogeneity in data and computing power, comprising: S10: Based on the network traffic data packets of each client node, extract the network traffic characteristics of each client node to construct the instruction fine-tuning dataset of each client node; calculate the maximum number of model parameters that each client node can bear based on the available computing resources of each client node to obtain the local model pruning rate of each client node. S20: Client nodes with the same local model pruning rate are grouped into a federated learning group, and each client node in the federated learning group is assigned the same structured pruned network traffic classification model. S30: Use the instructions of each client node to fine-tune the dataset to perform local iterative training of the network traffic classification model assigned to each client node, and calculate the local model aggregation parameter increment of each federated learning group after H local iterations. S40: Perform channel-aligned parameter mapping and aggregation on the local model aggregation parameter increments of all federated learning groups to obtain the global model parameter increments of the network traffic classification model, thereby updating the parameters of the network traffic classification model; S50: Determine whether the network traffic classification error of the network traffic classification model has converged after the parameter update. If it has not converged, return to step S30 to perform a new round of local iterative training. If it has converged, the trained network traffic classification model is obtained to classify the network traffic of each client node.

[0008] Preferably, based on the network traffic data packets of each client node, the network traffic characteristics of each client node are extracted to construct the instruction fine-tuning dataset for each client node, including: Generate standard pcap format files based on network traffic data packets from client nodes; The TShark command-line tool of Wireshark is used to parse standard pcap format files and extract network traffic characteristics; Network traffic characteristics are converted into a JSON-formatted instruction fine-tuning dataset using preset prompt word templates.

[0009] Preferably, the network traffic classification model assigned to each client node is iteratively trained locally using the fine-tuning dataset based on the instructions of each client node, including: Freeze the original weight matrix of the network traffic classification model assigned to the client node, and use the product of two low-rank matrices to represent the weight update matrix; The data in the instruction fine-tuning dataset is input into the network traffic classification model assigned to the client node. Based on the output of the network traffic classification model, the model parameters at the current local iteration, and the client-group correction term of the federated learning group to which the client node belongs during the previous local iteration training, the model parameters at the next local iteration are calculated until the number of local iterations equals H. The model parameters include at least the weight update matrix.

[0010] Preferably, the formula for calculating the model parameters in the next local iteration is: , , in, This represents the model parameters of the i-th client node during the h+1-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication. This represents the model parameters of the i-th client node during the h-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication. Indicates the local learning rate; This represents the stochastic gradient of the network traffic classification model assigned to the i-th client node. This represents the data batch of the i-th client node during the h-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication; This represents the client-group correction term of the federated learning group to which the i-th client node belongs during the local iterative training in round t-1 and the communication within the federated learning group in round e. This represents the set of client nodes in the g-th federated learning group; Let represent the average gradient of the network traffic classification model assigned to the g-th federated learning group.

[0011] Preferably, after step S30, the method further includes: Calculate the inner product of the local model aggregate parameter increment of each federated learning group and the global model parameter increment during the previous round of local iteration training, and select the local model aggregate parameter increments of federated learning groups with an inner product less than 0 as the local model aggregate parameter increments to be updated. The local model aggregate parameter increment to be updated is projected onto the normal plane of the global model parameter increment during the previous round of local iteration training to obtain the projected local model aggregate parameter increment. Using the norm scaling mechanism, the L2 norm ratio of the global weighted average update before and after projection is calculated based on the parameter increments and corresponding data volumes of the local models of each federated learning group. The updated local model aggregate parameter increment is obtained by multiplying the incremental local model aggregate parameter by the L2 norm ratio.

[0012] Preferably, the formula for calculating the increment of the aggregation parameters of the projected local model is: , in, This represents the increment of the projected local model aggregation parameters of the g-th federated learning group during the t-th round of local iterative training; This represents the increment of the local model aggregate parameters of the g-th federated learning group during the t-th round of local iterative training. , Indicates the inner product; This represents the increment of global model parameters during the (t-1)th round of local iterative training; This indicates a pre-defined positive number to prevent the denominator from being zero; Represents the square of the L2 norm; L2 norm ratio of global weighted average update before and after projection The calculation formula is: , in, This represents the total number of data points in the instruction fine-tuning dataset across all client nodes within the g-th federated learning group; Represents the L2 norm; The updated local model aggregation parameter increment is expressed as follows: , in, This represents the increment of the projected local model aggregation parameters of the g-th federated learning group during the t-th round of local iterative training after the update.

[0013] Preferably, channel-aligned parameter mapping and aggregation are performed on the local model aggregation parameter increments of all federated learning groups to obtain the global model parameter increments of the network traffic classification model, including: The standard federated averaging algorithm is used to aggregate parameter increments based on the local models of all federated learning groups to calculate the parameter increments of the unpruned network traffic classification model part. Based on the set of channel indices retained when allocating structured pruned network traffic classification models to each federated learning group, the local model aggregation parameters of each retained channel are incrementally restored to the global space through the index mapping function. For each channel in the pruned network traffic classification model, the parameter increment of each channel is calculated based on the local model aggregation parameter increments of all federated learning groups that retain that channel and are restored to the global space. Thus, the parameter increment of the pruned network traffic classification model is obtained based on the parameter increments of all channels in the pruned network traffic classification model. The global parameter increments of the network traffic classification model are obtained based on the parameter increments of the unpruned part of the network traffic classification model and the parameter increments of the pruned part of the network traffic classification model.

[0014] Preferably, the parameter increment of the unpruned network traffic classification model portion The calculation formula is: , in, This represents the total number of data points in the instruction fine-tuning dataset across all client nodes within the g-th federated learning group; This represents the local model aggregation parameter increment of the unpruned network traffic classification model part corresponding to the g-th federated learning group after the t-th round of local iterative training; The formula for calculating the parameter increment for each channel in the pruned network traffic classification model is as follows: , , in, The t-th iteration of the local iterative training of the network traffic classification model represents the pruned part of the model. The parameter increment of the first low-rank matrix of each channel; The t-th iteration of the local iterative training of the network traffic classification model represents the pruned part of the model. The parameter increment of the second low-rank matrix of each channel; , This represents the set of channel indices retained by the g-th federated learning group when assigning the structured, pruned network traffic classification model. This represents the total number of channels in the global model. This represents the parameter increment of the first low-rank matrix in the global space after the t-th round of local iterative training; This represents the parameter increment of the second low-rank matrix in the global space after the t-th round of local iterative training; Represents the mapping function, i.e., the first... The row / column position of each channel in the local model aggregation parameter increment of the g-th federated learning group.

[0015] Preferably, the formula for updating the parameters of the network traffic classification model is: , in, This represents the global model state of the network traffic classification model during the (t+1)th round of local iterative training; This represents the global model state of the network traffic classification model during the t-th round of local iterative training; This represents the server-side learning rate; This represents the global model parameter increment during the t-th round of local iterative training.

[0016] This invention also provides a federated network traffic classification system for heterogeneous data and computing power, comprising: The data acquisition and processing module is used to extract network traffic characteristics of each client node based on the network traffic data packets of each client node, thereby constructing the instruction fine-tuning dataset of each client node; and to calculate the maximum number of model parameters that each client node can bear based on the available computing resources of each client node, thereby obtaining the local model pruning rate of each client node. The model pruning and allocation module is used to group client nodes with the same local model pruning rate into a federated learning group, and to allocate the same structured pruned network traffic classification model to each client node in each federated learning group. The local iterative training module is used to fine-tune the dataset using the instructions of each client node, perform local iterative training on the network traffic classification model assigned to each client node, and calculate the local model aggregation parameter increment of each federated learning group after H local iterations. The parameter aggregation and update module is used to perform channel-aligned parameter mapping and aggregation on the local model parameter increments of all federated learning groups to obtain the global model parameter increments of the network traffic classification model, thereby updating the parameters of the network traffic classification model. The model acquisition module is used to determine whether the network traffic classification error of the network traffic classification model has converged after the parameter update. If it has not converged, it returns to the step of executing the local iterative training module to carry out a new round of local iterative training. If it has converged, the trained network traffic classification model is obtained to classify the network traffic of each client node.

[0017] The federated network traffic classification method for heterogeneous data and computing power provided in this application has the following advantages: 1. By evaluating the available computing and storage resources of edge client nodes, the maximum model capacity is adaptively calculated and the pruning rate is determined, thereby performing structured channel pruning on the model. Nodes with the same pruning rate are assigned to the same federated learning group, eliminating network structure heterogeneity during local aggregation within the group from the source. Local iterative training is performed on the network traffic classification model assigned to each client node, and the increment of local model aggregation parameters for each federated learning group is calculated after H local iterations. Furthermore, to address the issue of inconsistent model parameter dimensions caused by differences in channel pruning degrees among different federated learning groups, channel-aligned alignment is performed within the original global parameter space. Parameter mapping and aggregation are performed to obtain global model parameter increments, thereby updating the parameters of the network traffic classification model. When the updated network traffic classification model reaches the convergence condition, the trained network traffic classification model is obtained. This application combines the client node resource status to adaptively set the model pruning rate, adapting to the computing power and storage capacity of heterogeneous devices and avoiding resource overload of weak clients. At the same time, by grouping with the same pruning rate, structural heterogeneity within the group is eliminated, and the problem of parameter dimension differences across groups is solved by channel alignment mapping, effectively suppressing the laggard effect. While ensuring that all client nodes participate in training, the model convergence speed is accelerated and the accuracy of network traffic classification is improved. 2. When the client node performs low-rank fine-tuning training of the model, it not only uses network traffic features for forward propagation and gradient calculation, but also innovatively introduces a client-group correction term based on the difference before and after model training. This correction term guides the individual update direction to the group average direction, effectively suppressing the model drift problem caused by the non-independent and identically distributed characteristics of network traffic data. 3. Before performing cross-group parameter increment global aggregation, the local updates uploaded by each group are compared with the historical global model parameter increments by inner product comparison. For update items with directional conflicts (inner product less than zero), the projection of the normal plane of the previous round of global model parameter increments is removed, and an L2 norm scaling mechanism is used to maintain an effective gradient update magnitude while eliminating conflicts. 4. To address the parameter dimension misalignment problem caused by different pruning groups, a precise index mapping function was designed. During global aggregation, gradient fragments from different subspaces are accurately backfilled to the corresponding channel positions of the original global model. The total number of actual samples covering a specific channel is used as the denominator for weighting, thus achieving high-precision reorganization of heterogeneous models. Attached Figure Description

[0018] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein... Figure 1 Flowchart of the federated network traffic classification method for dual heterogeneity of data and computing power provided for this application; Figure 2A schematic diagram of the architecture of the federated network traffic classification system for heterogeneous data and computing power provided in this application. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0020] Please see Figure 1 , Figure 1 The diagram shows a flowchart of the federated network traffic classification method for heterogeneous data and computing power provided in this application. The method includes steps S10 to S50: S10: Based on the network traffic data packets of each client node, extract the network traffic characteristics of each client node to construct the instruction fine-tuning dataset for each client node; calculate the maximum number of model parameters that each client node can bear based on the available computing resources of each client node to obtain the local model pruning rate of each client node.

[0021] Optionally, each client node can directly complete the construction of the instruction fine-tuning dataset and the calculation of the local model pruning rate locally, and then directly report the local model pruning rate to the server. This ensures that network traffic data is always local to the client, protecting network traffic privacy while strictly complying with data compliance requirements.

[0022] Furthermore, generating the instruction fine-tuning dataset includes steps 1-1 to 1-3: Step 1-1: Generate a standard pcap format file based on network traffic data packets from the client node.

[0023] Steps 1-2: Use Wireshark's TShark command-line tool to parse the standard pcap format file and extract network traffic characteristics. For example, network traffic characteristics include network and transport layer header information, timing characteristics, and payload characteristics, such as source / destination IP, port, TCP flags, and payload.

[0024] Steps 1-3: Convert network traffic characteristics into a JSON-formatted instruction fine-tuning dataset using preset prompt word templates.

[0025] S20: Form a federated learning group by grouping client nodes with the same local model pruning rate, and assign the same structured pruned network traffic classification model to each client node in each federated learning group.

[0026] Specifically, by grouping client nodes with the same local model pruning rate into the same federated learning group and assigning corresponding group aggregation nodes to each federated learning group to realize data computation within the group, all client nodes within the same federated learning group can have a completely consistent model network structure and parameter dimensions. The server only needs to receive the model parameters of each federated learning group and perform global aggregation, thus forming a hierarchical federated learning architecture.

[0027] S30: Use the instructions of each client node to fine-tune the dataset to perform local iterative training of the network traffic classification model assigned to each client node, and calculate the local model aggregation parameter increment of each federated learning group after H local iterations.

[0028] Furthermore, in this application, the model allocated to each client node is trained locally using LoRA fine-tuning, specifically including steps 2-1 to 2-2: Step 2-1: Freeze the original weight matrix of the network traffic classification model assigned to client nodes. The weight update matrix is ​​represented by the product of two low-rank matrices (first low-rank matrix A and second low-rank matrix B). .

[0029] The forward propagation calculation formula for the linear layer within the model assigned to the client node is as follows: , This represents the output of the model. This represents the input to the model.

[0030] Step 2-2: Input the data from the instruction fine-tuning dataset into the network traffic classification model assigned to the client node. Based on the output of the network traffic classification model, the model parameters at the current local iteration, and the client-group correction term of the federated learning group to which the client node belongs during the previous local iteration training, calculate the model parameters for the next local iteration, until the number of local iterations equals H. The model parameters include at least the weight update matrix.

[0031] Specifically, the formula for calculating the model parameters in the next local iteration is: , , in, This represents the model parameters of the i-th client node during the h+1-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication. This represents the model parameters of the i-th client node during the h-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication. Indicates the local learning rate; This represents the stochastic gradient of the network traffic classification model assigned to the i-th client node. This represents the data batch of the i-th client node during the h-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication; This represents the client-group correction term of the federated learning group to which the i-th client node belongs during the local iterative training in round t-1 and the communication within the federated learning group in round e. This represents the set of client nodes in the g-th federated learning group; Let represent the average gradient of the network traffic classification model assigned to the g-th federated learning group.

[0032] It should be noted that, since this application uses H local iterations as a single iteration round for the entire network traffic classification model, the update of model parameters during the t-th local iteration training depends on the client-group correction term calculated during the (t-1)-th local iteration training. Since this application treats client nodes with the same pruning rate as a federated learning group, by influencing the average gradient of the federated learning group to which the client node belongs through model training gradient and using it for model parameter updates, the bias caused by heterogeneity within the group can be reduced, and the conflict caused by model heterogeneity can be eliminated, thereby solving the model drift problem caused by data heterogeneity.

[0033] S40: Perform channel-aligned parameter mapping and aggregation on the local model aggregation parameter increments of all federated learning groups to obtain the global model parameter increments of the network traffic classification model, thereby updating the parameters of the network traffic classification model.

[0034] Specifically, after all client nodes in each federated learning group have completed H local iterations, they upload the incremental local model aggregation parameters of the federated learning group to the server so that the server can perform a global aggregation operation. To address the issue of inconsistent model parameter dimensions caused by differences in channel pruning levels among different federated learning groups, the server performs channel-aligned parameter mapping and aggregation within the original global parameter space.

[0035] Furthermore, step S40 includes steps 3-1 to 3-4: Step 3-1: Use the standard federated averaging algorithm to aggregate parameter increments based on the local models of all federated learning groups, and calculate the parameter increments of the unpruned network traffic classification model part.

[0036] Specifically, the parameter increments in the unpruned network traffic classification model. The calculation formula is: , in, This represents the total number of data points in the instruction fine-tuning dataset across all client nodes within the g-th federated learning group; This represents the local model aggregation parameter increment of the unpruned network traffic classification model part corresponding to the g-th federated learning group after the t-th round of local iterative training.

[0037] Step 3-2: Based on the set of channel indices retained when allocating structured pruned network traffic classification models to each federated learning group, the local model aggregation parameters of each retained channel are incrementally restored to the global space through the index mapping function.

[0038] Step 3-3: For each channel of the pruned network traffic classification model, calculate the parameter increment of each channel based on the local model aggregate parameter increments restored to the global space by all federated learning groups that retain that channel. Thus, based on the parameter increments of all channels in the pruned network traffic classification model, obtain the parameter increments of the pruned network traffic classification model.

[0039] Specifically, the formula for calculating the parameter increment for each channel in the pruned network traffic classification model is as follows: , , in, The t-th iteration of the local iterative training of the network traffic classification model represents the pruned part of the model. The parameter increment of the first low-rank matrix of each channel; The t-th iteration of the local iterative training of the network traffic classification model represents the pruned part of the model. The parameter increment of the second low-rank matrix of each channel; , This represents the set of channel indices retained by the g-th federated learning group when assigning the structured, pruned network traffic classification model. This represents the total number of channels in the global model. This represents the parameter increment of the first low-rank matrix in the global space after the t-th round of local iterative training; This represents the parameter increment of the second low-rank matrix in the global space after the t-th round of local iterative training; Represents the mapping function, i.e., the first... The row / column position of each channel in the local model aggregation parameter increment of the g-th federated learning group.

[0040] Steps 3-4: Based on the parameter increments of the unpruned network traffic classification model and the parameter increments of the pruned network traffic classification model, obtain the global model parameter increments of the network traffic classification model.

[0041] Specifically, the server precisely fills back the gradient fragments from different subspaces to their corresponding positions in the global space, resolving the parameter index misalignment problem caused by pruning, thereby obtaining the complete global model parameter increment. .

[0042] S50: Determine whether the network traffic classification error of the network traffic classification model has converged after the parameter update. If it has not converged, return to step S30 to perform a new round of local iterative training. If it has converged, the trained network traffic classification model is obtained to classify the network traffic of each client node.

[0043] Specifically, the formula for updating the parameters of the network traffic classification model is as follows: , in, This represents the global model state of the network traffic classification model during the (t+1)th round of local iterative training; This represents the global model state of the network traffic classification model during the t-th round of local iterative training; This represents the server-side learning rate; This represents the global model parameter increment during the t-th round of local iterative training.

[0044] At the same time, the server updates and stores the global model parameter increments for this round as historical global model parameter increments for use in the next round of local iteration training: .

[0045] Optionally, in some embodiments of this application, steps 4-1 to 4-4 are further included after step S30: Step 4-1: Calculate the inner product of the local model aggregate parameter increment of each federated learning group and the global model parameter increment during the previous round of local iteration training, and select the local model aggregate parameter increments of federated learning groups with an inner product less than 0 as the local model aggregate parameter increments to be updated.

[0046] Step 4-2: Project the local model aggregate parameter increment to be updated onto the normal plane of the global model parameter increment during the previous round of local iterative training to obtain the projected local model aggregate parameter increment.

[0047] Specifically, the formula for calculating the increment of the aggregated parameters of the projected local model is as follows: , in, This represents the increment of the projected local model aggregation parameters of the g-th federated learning group during the t-th round of local iterative training; This represents the increment of the local model aggregate parameters of the g-th federated learning group during the t-th round of local iterative training. , Indicates the inner product; This represents the increment of global model parameters during the (t-1)th round of local iterative training; This indicates a preset positive number to prevent the denominator from being zero; This represents the square of the L2 norm.

[0048] Step 4-3: Using the norm scaling mechanism, based on the local model aggregation parameter increments and corresponding data volume of each federated learning group, calculate the L2 norm ratio of the global weighted average update before and after projection.

[0049] Specifically, the L2 norm ratio updated by the global weighted average before and after projection. The calculation formula is: , in, This represents the total number of data points in the instruction fine-tuning dataset across all client nodes within the g-th federated learning group; This represents the L2 norm.

[0050] Step 4-4: Based on the product of the incremental local model aggregate parameters and the L2 norm ratio, obtain the updated local model aggregate parameter increment.

[0051] Specifically, the updated local model aggregation parameter increment is expressed as: , in, This represents the increment of the projected local model aggregation parameters of the g-th federated learning group during the t-th round of local iterative training after the update.

[0052] It should be noted that steps 4-1 to 4-4, global conflict detection and elimination, are usually performed after the third round of local iterative training. Specifically, when the inner product of the local model aggregate parameter increment of the g-th federated learning group and the global model parameter increment of the previous round of local iterative training is less than 0, it indicates that the model parameter update direction of the federated learning group is contrary to the global model parameter increment direction of the previous round. Therefore, it is necessary to perform projection elimination on the update of the local model aggregate parameter increment of the federated learning group, so as to maintain the effective update magnitude of the gradient while eliminating the conflict between the local and global update directions.

[0053] This application significantly reduces the memory usage and computational overhead of edge client nodes by combining resource assessment with an adaptive structured pruning mechanism and low-rank fine-tuning techniques, making it possible to train large language models on resource-constrained real-world network edge devices. Simultaneously, the hierarchical federated architecture design, which uses homogeneous grouping based on model pruning rates, ensures that each federated learning group has a completely consistent network structure and parameter dimensions, fundamentally eliminating model heterogeneity barriers during local aggregation within the group and improving the overall system efficiency. Furthermore, addressing the non-independent and identically distributed nature of network traffic data among different client nodes, this application innovatively introduces a client-group correction term in local iterative training. This mechanism extracts and compensates for gradient bias by back-calculating the parameter differences before and after local iterative training, smoothly guiding the optimization direction of individual client nodes to the average gradient direction of their group. This effectively eliminates interference caused by local data skew and significantly alleviates intra-group model drift caused by data heterogeneity. Furthermore, to address the issue of conflicting gradient update directions that easily arise during global aggregation of groups with different pruning degrees, this application proposes a conflict detection and elimination mechanism based on historical global model parameter increments. When a conflict is detected between the local update direction of a group and the previous round of global model parameter increment direction, the conflicting component is eliminated by projecting onto the normal plane of the previous round of global model parameter increment. This is further enhanced by an introduced L2 norm scaling mechanism, which successfully eliminates the mutual cancellation effect caused by negative conflicts while maintaining the effective update step size of the original gradient, thereby accelerating the stable convergence of the global model. Finally, to address the issue of inconsistent model parameter dimensions uploaded by different groups in the hierarchical architecture, this application designs a channel-aligned heterogeneous parameter mapping and aggregation strategy. By constructing a precise index mapping function, gradient fragments from different pruning subspaces are accurately backfilled to the corresponding channel positions in the original global parameter space. Simultaneously, during aggregation, dynamic weighting is performed based on the actual total number of samples covering specific channels, maximizing the retention and utilization of the effective learning increments of each heterogeneous client and ensuring the integrity of the global model reorganization after pruning.

[0054] The following examples further illustrate the above-mentioned federated network traffic classification method oriented towards dual heterogeneity of data and computing power: Example 1 of this application details the parameter configuration and execution process of the above method in a specific software and hardware environment: 1. Deploy the initial open-source large language model (e.g., Llama 3 8B, with 8 billion parameters) on the server side; set up global communication rounds. Intra-group communication interval Server-side learning rate and local learning rate As an exemplary experimental configuration, the server-side used two Intel Xeon Gold 6348 processors, 512GB of RAM, and four NVIDIA A100 80GB GPUs. The operating system was Ubuntu 22.04, CUDA version 12.1, and PyTorch version 2.1.2. The base model was Llama 3 8B, and the model weights were loaded with BF16 precision. To form 140 heterogeneous client nodes, the maximum available GPU memory was limited to 80GB, 48GB, 32GB, and 24GB respectively using a logical client approach, corresponding to 20, 30, 40, and 50 client nodes respectively. Clients with 24GB of GPU memory were assigned to the low-resource group to avoid overlapping group boundaries. Each experiment was run three times independently using a random seed of 42, and the results were the average of the three runs.

[0055] 2. Data Processing: Each client node uses the tcpdump tool to capture real traffic data of the local network port and saves it as a .pcap file; the source / destination IP, port, TCP flags and payload are extracted using Wireshark's TShark command-line tool; and the dataset is converted into JSON format using a preset prompt word template for fine-tuning the command dataset.

[0056] 3. Resource Assessment and Grouping: The system contains 140 client nodes. Each node monitors its local GPU memory and reports its resource status to the server. The server then divides the nodes into four federated learning groups based on resource limitations. High resource group (video memory) 80GB): 100% of the model channels are retained (0% pruning rate), and the number of nodes is 20.

[0057] Medium-high resource group (40GB-80GB VRAM): 75% of model channels are retained (25% pruning rate), and the number of nodes is 30.

[0058] Medium resource group (24GB-40GB VRAM): 55% of model channels are retained (45% pruning rate), and the number of nodes is 40.

[0059] Low resource group (video memory) 24GB: 35% of the model channels are retained (65% pruning rate), and the number of nodes is 50.

[0060] Taking Llama 3 8B as an example, this model contains 32 Transformer layers, 32 query attention heads, 8 key-value attention heads, and 14,336 feedforward network intermediate channels. The server uses LLM-Pruner for structured channel pruning, treating attention head groups and feedforward network intermediate channels as pruning units, and ensuring synchronous adjustment of related projection layers through dependency analysis. For example, in the medium-high resource group, 24 query attention heads, 6 key-value attention heads, and 10,752 feedforward network intermediate channels can be retained; in the medium resource group, approximately 16 query attention heads, 4 key-value attention heads, and approximately 7,936 feedforward network intermediate channels can be retained; and in the low resource group, approximately 12 query attention heads, 3 key-value attention heads, and approximately 5,120 feedforward network intermediate channels can be retained.

[0061] 4. Local Fine-tuning: In local LoRA training, set the rank of the low-rank matrix. scaling factor The client freezes the pre-trained weights and only injects the Query and Value projection matrices in the attention mechanism into the LoRA module for training; a client-side correction term is introduced to correct the local gradient.

[0062] The exemplary training parameter settings are as follows: the maximum number of global communication rounds T is 50 rounds, the number of local iterations per round H is 100, and intra-group aggregation is performed once every 20 local iterations; the local learning rate is 2×10. -4 The server-side global update learning rate is 1.0, the optimizer is AdamW, the weight decay coefficient is 0.01, and the learning rate warm-up ratio is 0.0003. The single-client batch size is 2, the gradient accumulation steps are 8, the equivalent batch size is 32, the maximum input sequence length is 512, and the gradient clipping threshold is 1.0. The rank r of the LoRA low-rank matrix is ​​set to 8, the scaling factor alpha is set to 16, and dropout is set to 0.05. The LoRA module is injected only into the Query and Value projection matrices. If the relative rate of change of the validation loss in three consecutive rounds of global communication is less than 1.0, the optimization is successful. If the condition is met, the model is considered converged; if the condition is not met after 50 rounds, training is terminated and the global model with the lowest validation loss is output.

[0063] 5. Global Conflict Detection and Aggregation: After the 3rd round of global communication ( The server detected a negative inner product between the update direction of the low resource group and the historical global model parameter increment, i.e. The server immediately projects the update group onto the normal plane and applies the L2 norm scaling mechanism to restore the step size; then it performs heterogeneous parameter aggregation aligned by channel, updates the global model, and starts the next iteration until the loss function converges.

[0064] The collision detection numerical process in an exemplary run is as follows: After the third round of global communication, the LoRA parameter increments of each group are mapped to a unified global space. The L2 norm of the previous round's global model parameter increment is 0.842, and the L2 norm of the low-resource group's local model aggregate parameter increment is 0.571. Their inner product is -0.093, corresponding to a cosine similarity of -0.193 and an angle of approximately 101.2°. Therefore, it is determined that the update of this group conflicts with the direction of the previous round's global model parameter increment. The projection coefficient is calculated to be 0.1312 according to the normal plane projection formula. After projection, the inner product of the update of this group and the previous round's global model parameter increment is... The L2 norm of the projection update is 0.560. The L2 norms of the global weighted average update before and after projection are 0.462 and 0.438, respectively, resulting in a norm scaling ratio of 1.055. The L2 norm of the low-resource group update after scaling is 0.591, and it remains approximately orthogonal to the historical global update direction. After completing the fourth round of global updates using this conflict elimination result, the validation set loss decreased from 0.428 without conflict elimination to 0.391, and the Macro-F1 score increased from 92.8% to 93.4%, indicating that normal plane projection and norm scaling can maintain an effective update step size while removing conflict components.

[0065] Classification Performance Validation: The Llama 3 8B model was trained using the method proposed in this application on various network traffic test sets (USTC-TFC, ISCX-VPN, ISCX-Tor), and the trained network traffic classification model was tested. Test results show that the classification accuracy of the proposed method is approximately 94.2%, the macro-average precision is approximately 94.0%, the macro-average recall is approximately 93.8%, and the Macro-F1 score is approximately 93.9%. Furthermore, resource evaluation and structured pruning enable low-resource clients with 24GB of available GPU memory to participate in local training, demonstrating that the proposed method can maintain good network traffic classification performance and training efficiency while reducing the computational resource consumption of heterogeneous clients.

[0066] Example 2 of this application provides an application of the above method in a real industrial scenario of detecting abnormal network traffic in an enterprise park, to further clarify the feasibility of the method: In this application scenario, the server is deployed in the company's headquarters data center, and each client node is deployed in the edge gateway or branch server of each branch office. Due to the significant differences in the business types of each branch office (for example, the network traffic of the R&D department is mostly code submission and SSH protocol interaction, while the network traffic of the marketing department is mostly audio and video streams and external web browsing), the network traffic data captured by each node exhibits a highly non-independent and identically distributed characteristic in terms of feature distribution.

[0067] The specific implementation process and results are as follows: 1. Resource Adaptation and Privacy Protection in Local Iterative Training: Each branch's edge gateway (client node) first assesses the available computing resources on its local devices and, based on this assessment, obtains a lightweight large language model sub-network with the corresponding pruning rate from the headquarters server. Subsequently, within the local intranet environment, it extracts and reconstructs training data using its unique network traffic characteristics, and performs local model fine-tuning training incorporating client-side correction terms. Throughout the entire training cycle, the original network traffic data remains on the local campus network; the system only interacts with the server to update the model weight parameters, thereby improving model performance while strictly meeting enterprise-level data security and privacy compliance requirements.

[0068] 2. Resolution of cross-business line data heterogeneity conflicts: To address the gradient update conflicts caused by the highly heterogeneous traffic data between the R&D and Marketing departments, the headquarters server applies a conflict detection and elimination mechanism based on historical global model parameter increments before performing global model aggregation. Specifically, by detecting the inner product of the update increment of each edge gateway group and the direction of the previous round of global model parameter increments, and supplementing it with normal plane projection and norm scaling operations, the conflicting components of model parameter updates caused by different business flow characteristics are effectively eliminated, preventing model drift caused by heterogeneous data.

[0069] 3. Global Feature Fusion and Efficient Deployment: The server-side further completes the global model reorganization through a heterogeneous parameter aggregation mechanism aligned by channel; the final converged global large language model fully learns and integrates various abnormal traffic patterns across the entire enterprise (such as the internal network penetration test characteristics of the R&D department and the external DDoS attack characteristics of the marketing department). After the global model is distributed to each branch office, each resource-constrained edge gateway is equipped with high-precision local network anomaly detection, threat tracing, and automated security interpretation report generation capabilities based on the natural language large model.

[0070] Based on the federated network traffic classification method for heterogeneous data and computing power provided in the above embodiments, this application also provides a federated network traffic classification system for heterogeneous data and computing power, such as... Figure 2 As shown, the system specifically includes: The data acquisition and processing module 10 is used to extract the network traffic characteristics of each client node based on the network traffic data packets of each client node, thereby constructing the instruction fine-tuning dataset of each client node; and to calculate the maximum number of model parameters that each client node can bear based on the available computing resources of each client node, thereby obtaining the local model pruning rate of each client node.

[0071] The model pruning and allocation module 20 is used to group client nodes with the same local model pruning rate into a federated learning group, and to allocate the same structured pruned network traffic classification model to each client node in each federated learning group.

[0072] The local iterative training module 30 is used to fine-tune the dataset using the instructions of each client node, perform local iterative training on the network traffic classification model assigned to each client node, and calculate the local model aggregation parameter increment of each federated learning group after H local iterations.

[0073] The parameter aggregation and update module 40 is used to perform channel-aligned parameter mapping and aggregation processing on the local model aggregation parameter increments of all federated learning groups to obtain the global model parameter increments of the network traffic classification model, thereby updating the parameters of the network traffic classification model.

[0074] The model acquisition module 50 is used to determine whether the network traffic classification error of the network traffic classification model has converged after the parameter update. If it has not converged, it returns to the step of executing the local iterative training module to carry out a new round of local iterative training. If it has converged, the trained network traffic classification model is obtained to classify the network traffic of each client node.

[0075] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for classifying traffic in federated networks with heterogeneous data and computing power, characterized in that, include: S10: Based on the network traffic data packets of each client node, extract the network traffic characteristics of each client node, and thus construct the instruction fine-tuning dataset for each client node; The maximum number of model parameters that each client node can support is calculated based on the available computing resources of each client node, thereby obtaining the local model pruning rate of each client node. S20: Client nodes with the same local model pruning rate are grouped into a federated learning group, and each client node in the federated learning group is assigned the same structured pruned network traffic classification model. S30: Use the instructions of each client node to fine-tune the dataset to perform local iterative training of the network traffic classification model assigned to each client node, and calculate the local model aggregation parameter increment of each federated learning group after H local iterations. S40: Perform channel-aligned parameter mapping and aggregation on the local model aggregation parameter increments of all federated learning groups to obtain the global model parameter increments of the network traffic classification model, thereby updating the parameters of the network traffic classification model; S50: Determine whether the network traffic classification error of the network traffic classification model has converged after the parameter update. If it has not converged, return to step S30 to perform a new round of local iterative training. If convergence is achieved, a well-trained network traffic classification model is obtained to classify the network traffic of each client node.

2. The method for classifying federated network traffic in a heterogeneous data and computing power environment according to claim 1, characterized in that, Based on the network traffic data packets of each client node, the network traffic characteristics of each client node are extracted to construct a fine-tuning dataset for the instructions of each client node, including: Generate standard pcap format files based on network traffic data packets from client nodes; The TShark command-line tool of Wireshark is used to parse standard pcap format files and extract network traffic characteristics; Network traffic characteristics are converted into a JSON-formatted instruction fine-tuning dataset using preset prompt word templates.

3. The method for classifying federated network traffic in a heterogeneous data and computing power environment according to claim 1, characterized in that, The network traffic classification model assigned to each client node is trained locally iteratively using the dataset fine-tuned by the instructions of each client node, including: Freeze the original weight matrix of the network traffic classification model assigned to the client node, and use the product of two low-rank matrices to represent the weight update matrix; The data in the instruction fine-tuning dataset is input into the network traffic classification model assigned to the client node. Based on the output of the network traffic classification model, the model parameters at the current local iteration, and the client-group correction term of the federated learning group to which the client node belongs during the previous local iteration training, the model parameters at the next local iteration are calculated until the number of local iterations equals H. The model parameters include at least the weight update matrix.

4. The method for classifying federated network traffic in a heterogeneous data and computing power environment according to claim 3, characterized in that, The formula for calculating the model parameters in the next local iteration is: ， ， in, This represents the model parameters of the i-th client node during the h+1-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication. This represents the model parameters of the i-th client node during the h-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication. Indicates the local learning rate; This represents the stochastic gradient of the network traffic classification model assigned to the i-th client node. This represents the data batch of the i-th client node during the h-th local iteration in the t-th round of local iterative training and the e-th round of federated learning group communication; This represents the client-group correction term of the federated learning group to which the i-th client node belongs during the local iterative training in round t-1 and the communication within the federated learning group in round e. This represents the set of client nodes in the g-th federated learning group; Let represent the average gradient of the network traffic classification model assigned to the g-th federated learning group.

5. The method for classifying federated network traffic in a heterogeneous data and computing power environment according to claim 1, characterized in that, Step S30 is followed by: Calculate the inner product of the local model aggregate parameter increment of each federated learning group and the global model parameter increment during the previous round of local iteration training, and select the local model aggregate parameter increments of federated learning groups with an inner product less than 0 as the local model aggregate parameter increments to be updated. The local model aggregate parameter increment to be updated is projected onto the normal plane of the global model parameter increment during the previous round of local iteration training to obtain the projected local model aggregate parameter increment. Using the norm scaling mechanism, the L2 norm ratio of the global weighted average update before and after projection is calculated based on the parameter increments and corresponding data volumes of the local models of each federated learning group. The updated local model aggregate parameter increment is obtained by multiplying the incremental local model aggregate parameter by the L2 norm ratio.

6. The method for classifying federated network traffic in a heterogeneous data and computing power environment according to claim 5, characterized in that, The formula for calculating the increment of the aggregated parameters of the projected local model is: ， in, This represents the increment of the projected local model aggregation parameters of the g-th federated learning group during the t-th round of local iterative training; This represents the increment of the local model aggregate parameters of the g-th federated learning group during the t-th round of local iterative training. , Indicates the inner product; This represents the increment of global model parameters during the (t-1)th round of local iterative training; This indicates a pre-defined positive number to prevent the denominator from being zero; Represents the square of the L2 norm; L2 norm ratio of global weighted average update before and after projection The calculation formula is: ， in, This represents the total number of data points in the instruction fine-tuning dataset across all client nodes within the g-th federated learning group; Represents the L2 norm; The updated local model aggregation parameter increment is expressed as follows: ， in, This represents the increment of the projected local model aggregation parameters of the g-th federated learning group during the t-th round of local iterative training after the update.

7. The method for classifying federated network traffic in a heterogeneous data and computing power environment according to claim 1, characterized in that, Perform channel-aligned parameter mapping and aggregation on the local model aggregation parameter increments for all federated learning groups to obtain the global model parameter increments for the network traffic classification model, including: The standard federated averaging algorithm is used to aggregate parameter increments based on the local models of all federated learning groups to calculate the parameter increments of the unpruned network traffic classification model part. Based on the set of channel indices retained when allocating structured pruned network traffic classification models to each federated learning group, the local model aggregation parameters of each retained channel are incrementally restored to the global space through the index mapping function. For each channel in the pruned network traffic classification model, the parameter increment of each channel is calculated based on the local model aggregation parameter increments of all federated learning groups that retain that channel and are restored to the global space. Thus, the parameter increment of the pruned network traffic classification model is obtained based on the parameter increments of all channels in the pruned network traffic classification model. The global parameter increments of the network traffic classification model are obtained based on the parameter increments of the unpruned part of the network traffic classification model and the parameter increments of the pruned part of the network traffic classification model.

8. The method for classifying federated network traffic in a heterogeneous data and computing power environment according to claim 7, characterized in that, Parameter increments in the unpruned network traffic classification model The calculation formula is: ， in, This represents the total number of data points in the instruction fine-tuning dataset across all client nodes within the g-th federated learning group; This represents the local model aggregation parameter increment of the unpruned network traffic classification model part corresponding to the g-th federated learning group after the t-th round of local iterative training; The formula for calculating the parameter increment for each channel in the pruned network traffic classification model is as follows: ， ， in, The t-th iteration of the local iterative training of the network traffic classification model represents the pruned part of the model. The parameter increment of the first low-rank matrix of each channel; The t-th iteration of the local iterative training of the network traffic classification model represents the pruned part of the model. The parameter increment of the second low-rank matrix of each channel; , This represents the set of channel indices retained by the g-th federated learning group when assigning the structured, pruned network traffic classification model. This represents the total number of channels in the global model. This represents the parameter increment of the first low-rank matrix in the global space after the t-th round of local iterative training; This represents the parameter increment of the second low-rank matrix in the global space after the t-th round of local iterative training; Represents the mapping function, i.e., the first... The row / column position of each channel in the local model aggregation parameter increment of the g-th federated learning group.

9. The method for classifying federated network traffic in a heterogeneous data and computing power environment according to claim 1, characterized in that, The formula for updating the parameters of the network traffic classification model is: ， in, This represents the global model state of the network traffic classification model during the (t+1)th round of local iterative training; This represents the global model state of the network traffic classification model during the t-th round of local iterative training; This represents the server-side learning rate; This represents the global model parameter increment during the t-th round of local iterative training.

10. A federated network traffic classification system oriented towards dual heterogeneity of data and computing power, characterized in that, include: The data acquisition and processing module is used to extract the network traffic characteristics of each client node based on the network traffic data packets of each client node, thereby constructing the instruction fine-tuning dataset of each client node. The maximum number of model parameters that each client node can support is calculated based on the available computing resources of each client node, thereby obtaining the local model pruning rate of each client node. The model pruning and allocation module is used to group client nodes with the same local model pruning rate into a federated learning group, and to allocate the same structured pruned network traffic classification model to each client node in each federated learning group. The local iterative training module is used to fine-tune the dataset using the instructions of each client node, perform local iterative training on the network traffic classification model assigned to each client node, and calculate the local model aggregation parameter increment of each federated learning group after H local iterations. The parameter aggregation and update module is used to perform channel-aligned parameter mapping and aggregation on the local model parameter increments of all federated learning groups to obtain the global model parameter increments of the network traffic classification model, thereby updating the parameters of the network traffic classification model. The model acquisition module is used to determine whether the network traffic classification error of the network traffic classification model has converged after the parameter update. If it has not converged, it returns to the step of executing the local iterative training module to carry out a new round of local iterative training. If convergence is achieved, a well-trained network traffic classification model is obtained to classify the network traffic of each client node.

Images