Dynamic load balancing scheme with differential privacy for traffic analysis in safe crowdsourcing platform

Abstract

The invention discloses a differential privacy-based multi-dimensional dynamic load balancing scheduling scheme (DP-DMAPSA), which is constructed on a DPDK framework and is oriented to high-concurrency scenes such as a security crowdsourcing platform and the like. According to the scheme, multi-dimensional indexes such as queue depth, CPU utilization rate and processing delay are introduced, differential privacy mechanisms such as Laplacian and Gaussian are combined, noise disturbance is performed on load data, and reasoning attacks are prevented. In scheduling, the system dynamically selects a processing core according to a weighted hash value and a privacy index to realize flow consistency and load balancing. Meanwhile, a self-adaptive sub-flow granularity adjustment mechanism is designed, dynamic adjustment can be carried out along with flow and load changes, and the scheduling precision and the resource utilization rate are improved. Experimental results show that the scheme improves the system throughput while guaranteeing the data privacy, and is suitable for scenes of security crowdsourcing, data centers, edge computing and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network communication, in particular to an application for a security crowdsourcing platform, and a method and system for data packet processing, load balancing and data privacy protection in a high-throughput network environment. BACKGROUND

[0002] In a high-throughput network environment, especially in scenarios such as security crowdsourcing that need to process massive high-speed data streams, efficient and privacy-protected data packet processing is a key challenge. In order to effectively allocate network load in a multi-core processor environment, researchers have proposed various technical solutions. However, existing technologies still have deficiencies in performance, adaptability and privacy protection.

[0003] In terms of load balancing technology, traditional static methods have obvious limitations. For example, methods based on hash functions such as Equal-Cost Multi-Path (ECMP) routing are prone to "hash polarization", resulting in uneven load distribution. Classic scheduling algorithms such as Round-Robin (RR) and Highest Random Weight (HRW) also perform poorly in dealing with dynamic traffic fluctuations. Although the RR algorithm can disperse the load, it will disrupt the consistency of data streams, affecting data locality; while the HRW algorithm cannot adjust according to the real-time load of the core. In order to solve these problems, researchers such as Barbett have proposed RSS++, a receiver-side extension mechanism that can perceive load and state. RSS++ can dynamically adjust its hash function according to the real-time load of the CPU core, thereby achieving adaptive flow distribution. Experiments show that RSS++ can control the load imbalance between cores to within 5% while maintaining flow consistency. However, this method still faces challenges in real-time state monitoring, adjustment decision overhead, and completely eliminating load differences under complex traffic.

[0004] In the application of privacy protection in network systems, differential privacy technology has been widely studied. For example, Tavangaran et al. proposed a differential privacy enhancement scheme for federated learning scenarios in wireless networks. The scheme reduces the risk of privacy leakage through an optimized resource scheduling method and artificial noise injection, and experiments show that its scheduler can improve prediction accuracy by more than 6%. In addition, Wei et al. proposed a verifiable differential privacy framework that uses zero-knowledge proof to verify the correctness of noise generation without exposing the noise itself, thereby achieving a balance between data privacy and system transparency. Although existing research such as NetDPSyn has achieved a certain level of privacy protection for flow-level traffic statistics, it still faces challenges in terms of scalability and adaptability to complex network environments. Differential privacy protection, but it does not solve the dynamic scheduling problem. How to effectively combine differential privacy with high-performance load balancing is still an open challenge. In terms of adaptive scheduling technology, some research explores the use of prediction models to guide scheduling decisions. For example, Xu et al. use machine learning algorithms to predict traffic flow, capturing the spatiotemporal distribution characteristics of traffic flow through deep learning, which significantly outperforms traditional statistical methods in terms of prediction accuracy and efficiency. This study shows that learning-based prediction models can better adapt to complex and variable traffic patterns, providing strong decision support for adaptive scheduling strategies. In summary, although existing technologies have made progress in load balancing, privacy protection, and adaptive scheduling in a single direction, few solutions can organically combine the three to meet the needs of high performance, strong privacy, and dynamic adaptation. SUMMARY

[0005] The main purpose of the present application is to solve the problem that the prior art is difficult to balance high performance and privacy protection in processing high-throughput network traffic, especially in application scenarios such as secure crowdsourcing platforms. The present application provides a dynamic adaptive load balancing scheme named DP-DMAPSA, which combines differential privacy mechanism with multi-dimensional load perception scheduling, aiming to achieve efficient, robust and credible traffic analysis and processing.

[0006] The purpose of the present application is achieved by the following technical solutions: Step 1, in order to generate flow consistency parameters for scheduling decisions, the system extracts the five-tuple features of each data packet and generates a unique flow identifier through a hash function; the flow identifier will be used as a weight factor in the subsequent multi-dimensional weighting strategy, and will participate in the scheduling score calculation of the target processing core together with other noisy load indicators. Step 2, respectively add differential privacy noise to the real-time load indicators of multiple processing cores, where Laplace noise is added to queue depth and CPU usage, and Gaussian noise is added to processing delay; Step 3, based on the noisy load indicators and hash values, use a multi-dimensional weighting strategy to calculate the scheduling weight of the target core, and then select the scheduling target core through the exponential mechanism; Step 4, based on the current average queue depth and traffic intensity of the system, dynamically adjust the scheduling granularity to achieve adaptive sub-flow scheduling; Step 5, continuously update the load indicators and scheduling parameters during system operation to achieve collaborative optimization of load balancing and privacy protection; BRIEF DESCRIPTION OF DRAWINGS

[0007] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description only constitute some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0008] Figure 1 Fig. 1 is a schematic diagram of a DP-DMAPSA system architecture according to an embodiment of the present application. Figure 2 Fig. 2 is a flowchart of a DP-DMAPSA algorithm according to an embodiment of the present application. Specific implementation

[0009] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute some of the embodiments of the present application, but not all the embodiments of the present application, which does not constitute a limitation to the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0010] The dynamic load balancing scheduling scheme with differential privacy for traffic analysis in the safe crowd-sourcing platform described in the embodiments of the present application comprises the following steps: 1) Flow consistency parameter generation: the five-tuple features of the data packet are extracted, including the source address, the destination address, the source port, the destination port and the protocol type, and the five-tuple information is calculated by a hash function (such as CRC32) to generate a unique flow identifier; the flow identifier will be used as a weight factor in the subsequent multi-dimensional weighted strategy, and will participate in the scheduling score calculation of the target processing core together with the noisy load index, so as to integrate the flow features of the data packet into the differential privacy scheduling decision, and to realize flow consistency. 2) Differential privacy of load index: the system regularly obtains the load state of each processing core, including the number of data packets in the current input queue , the number of processed data packets per unit time , and the average processing delay , as the input index of the subsequent scheduling strategy. In order to realize differential privacy protection, the system adds noise disturbance to each load index, wherein 、 the index adds Laplace noise, because its sensitivity is 1, which is suitable for Laplace mechanism; the index adds Gaussian noise, and its sensitivity is defined according to the maximum single packet delay . The disturbed values are respectively represented as . 3) Weighted score and core selection: The system calculates a comprehensive scheduling score for each core, denoted as: where the weight formula is: The above weights collectively affect the scheduling priority score, is an acceleration factor at the initial stage of system startup, used to quickly converge the scheduling strategy. The system uses an exponential mechanism to select candidate processing cores, and according to the scheduling score calculated in step 5, it performs weighted sampling to finally determine a target core for processing the current data packet. This scheduling strategy as a whole meets the differential privacy requirement, avoiding the disclosure of core state information in the scheduling process. 4) Adaptive granularity adjustment: The system calculates the scheduling granularity based on the average queue depth and traffic intensity, where the load factor is defined as: The scheduling granularity function is represented in exponential mapping form as: where is the number of current batch data packets, and are hyperparameters that adjust the sensitivity, is the average queue length, is the maximum capacity. 5) Continuous optimization: During system operation, the above load indicator acquisition, parameter update, and scheduling decision will continue, forming a closed-loop collaborative optimization process to dynamically balance the load and continuously protect privacy. This paper proposes a differential privacy dynamic load balancing scheme for traffic analysis in a secure crowdsourcing platform to solve the problem of processing high-speed, dynamic data streams generated by "crowdsourcing" nodes while effectively protecting the confidentiality of system operation state. It ensures the high performance and stability of the platform backend when dealing with massive, sudden traffic shocks by fusing queue depth, CPU usage, and other multi-dimensional load indicators for adaptive scheduling. At the same time, the built-in differential privacy mechanism provides quantifiable security protection for sensitive information such as platform load, which is crucial for enhancing platform credibility, encouraging user participation, and building a trustworthy crowdsourcing ecosystem.

Claims

1. A dynamic load balancing scheduling scheme with differential privacy for traffic analysis in a secure crowd-sourcing platform, characterized in that, The method is suitable for high-throughput network traffic processing environment, comprising the following steps: Step 1, in order to generate flow consistency parameters for scheduling decision, the system extracts the five-tuple features of each data packet, and generates a unique flow identifier by hash function calculation; The flow identifier will be used as one of the weight factors in the subsequent multi-dimensional weighted strategy, and will participate in the scheduling score calculation of the target processing core together with other noisy load indicators. Step 2, respectively add differential privacy noise to the real-time load indicators of multiple processing cores, wherein Laplace noise is added to queue depth and CPU usage, and Gaussian noise is added to processing delay. Step 3, according to the noisy load indicators and hash values, adopt multi-dimensional weighted strategy to calculate the scheduling weight of target core, and then select the scheduling target core through exponential mechanism; Step 4, based on the current average queue depth and traffic intensity of the system, dynamically adjust the scheduling granularity to realize adaptive sub-flow scheduling; Step 5, continuously update the load indicators and scheduling parameters during the system running process, realize the collaborative optimization of load balancing and privacy protection.

2. The method of claim 1, wherein the method further comprises: The process of step 1 is specifically: Extract the five-tuple features of the data packet, which includes source address, destination address, source port, destination port and protocol type, and calculate the five-tuple information through hash function (such as CRC32) to generate a unique flow identifier; The flow identifier will be used as one of the weight factors in the subsequent multi-dimensional weighted strategy, and will participate in the scheduling score calculation of the target processing core together with other noisy load indicators.

3. The method of claim 1, wherein the method further comprises: The process of step 2 is specifically: The system periodically acquires the load status of each processing core, including the number of data packets in the current input queue , the number of processed data packets per unit time , and the average processing delay , as input indicators for subsequent scheduling strategies. To achieve differential privacy protection, the system adds noise disturbance to each load indicator, where , The indicator adds Laplace noise, which has a sensitivity of 1 and is suitable for the Laplace mechanism. The indicator adds Gaussian noise, and its sensitivity is determined by the maximum single packet delay The disturbed values are represented as .​ 4. The method of claim 1, wherein the method further comprises: The process of step 3 is specifically: The system calculates the comprehensive scheduling score of each core, denoted as: wherein the weight formula of each term is: The above weights jointly affect the scheduling priority score, is an acceleration factor at the initial stage of system startup, used for rapid convergence of the scheduling strategy. The system adopts an exponential mechanism to select a candidate processing core, and performs weighted sampling according to the scheduling score calculated in step 5 to finally determine a target core for processing the current data packet. The scheduling strategy as a whole meets - Differential privacy requirements to avoid revealing core state information in the scheduling process.

5. The method of claim 1, wherein the method further comprises: The process of step 4 is specifically: The system calculates the scheduling granularity according to the average queue depth and the traffic intensity, wherein the load factor is defined as: The scheduling granularity function is expressed in the form of exponential mapping as: Wherein N is the number of current batch data packets, and is a hyperparameter for adjusting sensitivity, is the average queue length, is the maximum capacity.

6. The method of claim 1, wherein the method further comprises: The process of step 5 is specifically: During the system running process, the above load indicator acquisition, parameter update and scheduling decision will be continuously carried out, forming a closed-loop collaborative optimization process to dynamically balance the load and continuously protect privacy.

Images