Packet classification parallel method and system based on ensemble learning

Abstract

The application provides a packet classification parallel method and system based on ensemble learning, comprising the following steps: S1, establishing a rule search cost model to quantify the complementarity between different instances of an algorithm; S2, using a genetic algorithm to generate an optimal combination scheme of different instances of the algorithm, and after finding an algorithm instance meeting the requirements, constructing an algorithm instance library; and S3, constructing a packet classification prediction model, and selecting a suitable algorithm instance from the constructed algorithm instance library to complete a packet classification operation. The application first acutely observes the worst case existing in the existing packet classification method and the cause thereof, and under the enlightenment of ensemble learning, mines the complementarity existing between different instances of an algorithm to optimize the performance.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a parallel method and system for packet classification based on ensemble learning. Background Technology

[0002] To meet the demands of new network applications, routers need to provide enhanced functionality, such as firewalls, policy-based routing, quality of service (QoS), and traffic metering. These services require classifying incoming data packets into different flows and then performing various operations based on the assigned flows. Therefore, packet classification is a fundamental building block and key technology for these functions.

[0003] The rapid development of the internet has led to an increased demand for sufficient bandwidth in the underlying infrastructure. The success of fiber optic technology has facilitated high-speed data transmission over communication links. This has also sparked an extreme pursuit of packet classification efficiency. In practical applications, when a large number of data packets are being transmitted, multiple replicas of the algorithm instance with the best overall performance are often deployed to perform parallel packet classification operations.

[0004] However, most packet classification algorithms prioritize fast lookup speed, sometimes at the expense of long-tail latency and neglecting overall performance stability. This leads to significant performance fluctuations in packet lookup time, primarily manifested in three aspects: First, a prominent long-tail phenomenon exists, with the 95th and 99th percentiles (P95 and P99) of the lookup time significantly higher than the mean. Second, the lookup time exhibits large variance, indicating poor overall performance stability. Third, performance fluctuations intensify with increasing rule set skewness. High skewness is a major factor contributing to worst-case performance. Therefore, existing packet classification algorithms suffer from a general worst-case scenario. This manifests as a significant increase in lookup time for certain packets. While intuitive multi-instance replica parallel solutions can linearly increase throughput, they still face performance fluctuations due to the unavoidable worst-case scenarios of individual algorithm instances. These performance fluctuations negatively impact the quality of service (QoS) of network applications.

[0005] Packet classification is a critical function on various network devices. Its main task is to find the highest-priority rule that matches an incoming packet within a rule set. Most existing methods focus on improving average lookup performance, but packet classification methods often encounter worst-case scenarios caused by rule set skew, leading to a long-tail problem. Therefore, when deploying multiple identical instances of the best-performing algorithm to increase throughput, the persistent long-tail problem significantly impacts the quality of service (QoS) of network applications, a prevalent issue.

[0006] Patent document CN106598747A discloses a parallel processing method and apparatus for network data packets, relating to the field of data processing technology. Its main objective is to improve the efficiency of parallel processing of network data packets. The method involves setting the length of a single variable in the read / write variable section of a session data structure entry corresponding to a network session to be no greater than the word length of the central processing unit (CPU). When at least two CPU cores receive different network data packets from the same network session, each CPU core acquires the session data structure entry corresponding to that network session. Based on the execution status of the network data packets within the CPU core, atomic operations are performed to modify the read / write variable section of the session data structure entry. This invention observes the worst-case scenario of existing packet classification methods and its causes, and, inspired by ensemble learning, explores the complementarity between different instances of the algorithm to achieve efficient and stable performance. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a parallel method and system for packet classification based on ensemble learning.

[0008] According to the present invention, a parallel method for packet classification based on ensemble learning includes:

[0009] Step S1: Establish a rule-based search cost model and quantify the complementarity between different instances of the algorithm;

[0010] Step S2: Use a genetic algorithm to generate the best combination of different instances of the algorithm. After finding the algorithm instances that meet the requirements, build an algorithm instance library.

[0011] Step S3: Construct a packet classification prediction model and select a suitable algorithm instance from the constructed algorithm instance library to complete the packet classification operation.

[0012] Preferably, in step S1:

[0013] A search cost quantification model based on rules is proposed to measure the complementarity between different instances of the algorithm. In order to determine complementary algorithm instances, from the perspective of rules, when considering a specific algorithm instance, the subset of rules that generate the highest search cost is determined. For complementary algorithm instances, their respective high-cost rule subsets have the least overlap.

[0014] The spatial partitioning step is represented as the internal nodes of a tree, and the partitioned subspaces are represented as leaf nodes. The cost of rule lookup is modeled from the perspective of rule insertion. The depth of reaching a leaf node in the tree structure represents the cost required to find a rule. During the construction process, the algorithm determines the optimal parameter configuration from the search space. Each parameter configuration corresponds to an algorithm instance. In the process of searching for the optimal parameter configuration, an additional thread is added to record the time cost of inserting rules for each algorithm instance. This constructs a rule lookup cost model for each algorithm instance, which serves as the data basis for measuring the performance and complementarity of different algorithm instances.

[0015] Preferably, in step S2:

[0016] After determining the cost quantification model based on given rules, an efficient optimal algorithm instance combination method based on genetic algorithm is proposed:

[0017] Reduce the number of candidate algorithm instances: simplify the permutation problem into a combinatorial problem; filter algorithm instances whose average rule search cost is higher than a preset standard; filter algorithm instances whose memory usage is higher than a preset standard.

[0018] Find a combination of algorithm instances that minimizes the overall rule cost, which can be abstracted into the following formula:

[0019]

[0020] Where Ω represents the total space of possible combinations, τ represents a specific combination, N is the number of rules, and cost[i][j] refers to the algorithm instance AI. i The search cost of rule j in the algorithm; AI is an abbreviation for Algorithm Instance.

[0021] Genetic algorithms are used to process rule sets and combinations of algorithm instances. In genetic algorithms, the fitness of each feasible solution is evaluated based on multiple metrics. The overall performance of each algorithm instance is represented by its average rule cost. i The average rule cost is stored in the array avgCost[i], and the reciprocal of avgCost[i] is used as the AI. i The fitness value, fitness[i], is shown in Formula (2). A higher fitness value corresponds to a lower average cost, and algorithm instances with higher fitness values ​​have a greater chance of being selected. Formula (3) is used to calculate the selection of AI. i The probability of probability[i] is used to sort the candidate algorithm instance set in descending order based on the fitness value. Sort the algorithm instances in the table:

[0022]

[0023]

[0024] The two-dimensional array `cost` stores the rule search cost corresponding to each algorithm instance, where `cost[i][j]` represents the AI. i The search cost of rule j is calculated, and the fitness and selection probability of each algorithm instance are calculated. The set of valid candidate algorithm instances is represented by N, which represents the total number of rules in the rule set. Each iteration round includes: evolution of candidate algorithm instances; updating of the algorithm instance library and acquisition of new combination schemes; and evaluation to determine whether the termination condition is met.

[0025] Preferably, the evolution process of candidate algorithm instances includes replication and crossover: A roulette wheel algorithm is used to determine the algorithm instance selected in each round. First, a random number from 0 to 1 is generated and compared sequentially with the cumulative probability of each algorithm instance. If the probability exceeds the random number, the corresponding algorithm instance is selected. The selection process is proportional to the probability defined in formula (3); Crossover is performed: The two selected algorithm instances are denoted as AI. j and AI k The two algorithms are combined to form a new AI′, where the cost of each rule in AI′ corresponds to the smaller of the two algorithm instances; the greater the complementarity between the different instances of the algorithm, the lower the cost of the newly obtained algorithm instance AI′. The fitness of AI′ is calculated and denoted as curFitness.

[0026] Updates to the algorithm instance library and acquisition of combinatorial schemes: If the current fitness value curFitness is greater than the previous fitness value preFitness, it means that the current crossover round has produced the best result to date, and the new AI' is considered from the set. Removed previous AI j and AI k Insert AI′, calculate the selection probability of AI′ and record its algorithm instance combination scheme;

[0027] Determine if the algorithm meets the termination conditions: The algorithm includes two termination conditions. First, when the number of iterations reaches a threshold ρ, the algorithm stops executing, setting an upper limit on the number of iterations. Second, when the difference between the fitness values ​​generated by two consecutive iterations is less than a threshold ε, the algorithm terminates. The algorithm ends execution when either of these conditions is met, obtaining the desired outcome. The combination of algorithm instances with the highest fitness is denoted as τ. * And return;

[0028] The algorithm has a time complexity of O(Nρ), where N and ρ represent the number of rules in the rule set and the number of iterations, respectively. The algorithm consists of three parts, which are executed in each iteration. First, the algorithm iterates through all rules to form AI′, with a complexity of O(N). Second, the algorithm updates AI by deleting and inserting rules. The complexity of the operation is expressed as The third part involves checking the termination condition, implemented with O(1) complexity; the algorithm iterates at most ρ rounds, with a total complexity of O(1). Given Furthermore, experimental results show that the algorithm typically converges in less than 5 iterations, thus the overall complexity is approximately O(N).

[0029] Preferably, in step S3:

[0030] A linear regression prediction model is proposed, which selects the optimal algorithm instance for each incoming data packet while considering load balancing.

[0031] An incremental training strategy is used to address the cold start problem; initially, an algorithm instance is randomly selected from the library for packet classification, and each packet P is associated with a triplet.<P,AI,t> The related features are: AI represents the selected algorithm instance, t represents the search time; a preset number of samples are accumulated, the prediction model selects the algorithm instance for data packet classification, the prediction model performs incremental updates to adapt to distribution drift and improve the overall robustness of the solution;

[0032] The linear regression model is represented as follows:

[0033] y=α0+α1x1+α2x2+α3x3 (4)

[0034] Where y represents the estimated message lookup time, x i Indicates the extracted bytes, α i This represents the parameters that the model needs to learn;

[0035] The algorithm instances whose workloads are in a preset saturation state are dynamically masked. A queue is established for each algorithm instance in the repository. When there are waiting data packets in the queue, the corresponding algorithm instance is temporarily marked as unavailable, and the remaining unsaturated algorithm instances are available for selection.

[0036] According to the present invention, a parallel packet classification system based on ensemble learning includes:

[0037] Module M1: Establishes a rule-based search cost model and quantifies the complementarity between different instances of the algorithm;

[0038] Module M2: Uses a genetic algorithm to generate the best combination of different instances of the algorithm, and builds an algorithm instance repository after finding algorithm instances that meet the requirements;

[0039] Module M3: Constructs a data packet classification cost prediction model and selects appropriate algorithm instances from the constructed algorithm instance library to complete the data packet classification operation.

[0040] Preferably, in module M1:

[0041] A search cost quantification model based on rules is proposed to measure the complementarity between different instances of the algorithm. In order to determine complementary algorithm instances, from the perspective of rules, when considering a specific algorithm instance, the subset of rules that generate the highest search cost is determined. For complementary algorithm instances, their respective high-cost rule subsets have the least overlap.

[0042] The spatial partitioning step is represented as the internal nodes of a tree, and the partitioned subspaces are represented as leaf nodes. The cost of rule lookup is modeled from the perspective of rule insertion. The depth of reaching a leaf node in the tree structure represents the cost required to find a rule. During the construction process, the algorithm determines the optimal parameter configuration from the search space. Each parameter configuration corresponds to an algorithm instance. In the process of searching for the optimal parameter configuration, an additional thread is added to record the time cost of inserting rules for each algorithm instance. This constructs a rule lookup cost model for each algorithm instance, which serves as the data basis for measuring the performance and complementarity of different algorithm instances.

[0043] Preferably, in module M2:

[0044] After determining the cost quantification model based on given rules, an efficient optimal algorithm instance combination method based on genetic algorithm is proposed:

[0045] Reduce the number of candidate algorithm instances: simplify the permutation problem into a combinatorial problem; filter algorithm instances whose average rule search cost is higher than a preset standard; filter algorithm instances whose memory usage is higher than a preset standard.

[0046] Find a combination of algorithm instances that minimizes the overall rule cost, which can be abstracted into the following formula:

[0047]

[0048] Where Ω represents the total space of possible combinations, τ represents a specific combination, N is the number of rules, and cost[i][j] refers to the algorithm instance AI. i The search cost of rule j in the algorithm; AI is an abbreviation for Algorithm Instance.

[0049] Genetic algorithms are used to process rule sets and combinations of algorithm instances. In genetic algorithms, the fitness of each feasible solution is evaluated based on multiple metrics. The overall performance of each algorithm instance is represented by its average rule cost. i The average rule cost is stored in the array avgCost[i], and the reciprocal of avgCost[i] is used as the AI. iThe fitness value, fitness[i], is shown in Formula (2). A higher fitness value corresponds to a lower average cost, and algorithm instances with higher fitness values ​​have a greater chance of being selected. Formula (3) is used to calculate the selection of AI. i The probability of probability[i] is used to sort the candidate algorithm instance set in descending order based on the fitness value. Sort the algorithm instances in the table:

[0050]

[0051]

[0052] The two-dimensional array `cost` stores the rule search cost corresponding to each algorithm instance, where `cost[i][j]` represents the AI. i The search cost of rule j is calculated, and the fitness and selection probability of each algorithm instance are calculated. The set of valid candidate algorithm instances is represented by N, which represents the total number of rules in the rule set. Each iteration round includes: evolution of candidate algorithm instances; updating of the algorithm instance library and acquisition of new combination schemes; and evaluation to determine whether the termination condition is met.

[0053] Preferably, the evolution process of candidate algorithm instances includes replication and crossover: A roulette wheel algorithm is used to determine the algorithm instance selected in each round. First, a random number from 0 to 1 is generated and compared sequentially with the cumulative probability of each algorithm instance. If the probability exceeds the random number, the corresponding algorithm instance is selected. The selection process is proportional to the probability defined in formula (3); Crossover is performed: The two selected algorithm instances are denoted as AI. j and AI k The two algorithms are combined to form a new AI′, where the cost of each rule in AI′ corresponds to the smaller of the two algorithm instances; the greater the complementarity between the different instances of the algorithm, the lower the cost of the newly obtained algorithm instance AI′. The fitness of AI′ is calculated and denoted as curFitness.

[0054] Updates to the algorithm instance library and acquisition of combinatorial schemes: If the current fitness value curFitness is greater than the previous fitness value preFitness, it means that the current crossover round has produced the best result to date, and the new AI' is considered from the set. Removed previous AI j and AI k Insert AI′, calculate the selection probability of AI′ and record its algorithm instance combination scheme;

[0055] Determine if the algorithm meets the termination conditions: The algorithm includes two termination conditions. First, when the number of iterations reaches a threshold ρ, the algorithm stops executing, setting an upper limit on the number of iterations. Second, when the difference between the fitness values ​​generated by two consecutive iterations is less than a threshold ε, the algorithm terminates. The algorithm ends execution when either of these conditions is met, obtaining the desired outcome. The combination of algorithm instances with the highest fitness is denoted as τ. * And return;

[0056] The algorithm has a time complexity of O(Nρ), where N and ρ represent the number of rules in the rule set and the number of iterations, respectively. The algorithm consists of three parts, which are executed in each iteration. First, the algorithm iterates through all rules to form AI′, with a complexity of O(N). Second, the algorithm updates AI by deleting and inserting rules. The complexity of the operation is expressed as The third part involves checking the termination condition, implemented with O(1) complexity; the algorithm iterates at most ρ rounds, with a total complexity of O(1). Given Furthermore, experimental results show that the algorithm typically converges in less than 5 iterations, thus the overall complexity is approximately O(N).

[0057] Preferably, in module M3:

[0058] A linear regression prediction model is proposed, which selects the optimal algorithm instance for each incoming data packet while considering load balancing.

[0059] An incremental training strategy is used to address the cold start problem; initially, an algorithm instance is randomly selected from the library for packet classification, and each packet P is associated with a triplet.<P,AI,t> The related features are: AI represents the selected algorithm instance, t represents the search time; a preset number of samples are accumulated, the prediction model selects the algorithm instance for data packet classification, the prediction model performs incremental updates to adapt to distribution drift and improve the overall robustness of the solution;

[0060] The linear regression model is represented as follows:

[0061] y=α0+α1x1+α2x2+α3x3 (4)

[0062] Where y represents the estimated message lookup time, x i Indicates the extracted bytes, α i This represents the parameters that the model needs to learn;

[0063] The algorithm instances whose workloads are in a preset saturation state are dynamically masked. A queue is established for each algorithm instance in the repository. When there are waiting data packets in the queue, the corresponding algorithm instance is temporarily marked as unavailable, and the remaining unsaturated algorithm instances are available for selection.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] 1. This invention first keenly observes the worst-case scenario of existing packet classification methods and the reasons for its occurrence, and, inspired by ensemble learning, explores the complementarity between different instances of the algorithm;

[0066] 2. This invention addresses the following three challenges: how to develop a general model to measure the complementarity between different instances of an algorithm, how to efficiently determine the optimal combination of algorithm instances in a large search space, and how to select a suitable algorithm instance to perform a search operation for each incoming data packet when an algorithm instance library is provided.

[0067] 3. This invention fully recognizes and utilizes the complementarity between different instances of the algorithm, thereby achieving enhanced stability and improved throughput; to the best of our knowledge, this research problem and the solution involved are novel in existing research. Attached Figure Description

[0068] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0069] Figure 1 This is an overall framework diagram of the present invention;

[0070] Figure 2 The flowchart shows the optimal algorithm instance combination scheme of the present invention. Detailed Implementation

[0071] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0072] Example 1:

[0073] This invention relates to packet classification methods in networks, and proposes a parallel optimization framework that can achieve high efficiency and stable performance, applicable to various existing packet classification methods.

[0074] This invention proposes a novel solution for parallel packet classification based on ensemble learning, aiming to achieve efficient and stable performance. This solution utilizes the principles of ensemble learning to generate an optimal combination of diverse algorithm instances that exhibit similar performance and complementary characteristics. These instances collaborate with each other to achieve efficient and stable performance. Addressing the demand for high-quality services in networks, this invention proposes an optimized framework for ensemble packet classification, which can significantly improve the stability and throughput of packet classification methods.

[0075] This invention discloses a parallel optimization framework for packet classification with high efficiency and stability suitable for small and medium-sized network devices, and its construction method.

[0076] The ever-growing demands of new network applications require routers to provide enhanced functionalities, such as traffic metering and quality of service, all of which rely on packet classification. Simultaneously, thanks to the significant increase in network transmission speeds, multiple algorithm instances with optimal overall performance are typically deployed for parallel packet classification to optimize throughput. While this simple solution can linearly improve throughput, it cannot eliminate performance fluctuations present in each identical algorithm instance. This is because most algorithms prioritize optimizing average search speed, often neglecting overall performance stability and the long-tail problem. Addressing the shortcomings of existing packet classification algorithms in terms of performance stability, this invention proposes an ensemble parallel framework for packet classification as a solution, aiming to further enhance and stabilize its performance by addressing the long-tail problem. Inspired by ensemble learning, the ensemble parallel packet classification is based on the concept that different algorithm instances exhibit diverse worst-case distributions, thus complementing each other. To quantify the complementarity between different algorithm instances, this invention first constructs a general rule search cost model, providing a foundation for analyzing the complementarity of different instances. Subsequently, this invention employs a genetic algorithm to generate the optimal combination of algorithm instances. Furthermore, this invention introduces a lightweight prediction model to select the algorithm instance with the shortest processing time to handle packets, while considering load balancing. By fully leveraging the complementarity between different instances of the algorithm, the impact of long tails and worst-case scenarios can be significantly mitigated, improving performance efficiency and stability.

[0077] According to the present invention, a parallel method for packet classification based on ensemble learning is provided, such as... Figures 1-2 As shown, it includes:

[0078] Step S1: Establish a rule-based search cost model and quantify the complementarity between different instances of the algorithm;

[0079] Specifically, in step S1:

[0080] A search cost quantification model based on rules is proposed to measure the complementarity between different instances of the algorithm. In order to determine complementary algorithm instances, from the perspective of rules, when considering a specific algorithm instance, the subset of rules that generate the highest search cost is determined. For complementary algorithm instances, their respective high-cost rule subsets have the least overlap.

[0081] The spatial partitioning step is represented as the internal nodes of a tree, and the partitioned subspaces are represented as leaf nodes. The cost of rule lookup is modeled from the perspective of rule insertion. The depth of reaching a leaf node in the tree structure represents the cost required to find a rule. During the construction process, the algorithm determines the optimal parameter configuration from the search space. Each parameter configuration corresponds to an algorithm instance. In the process of searching for the optimal parameter configuration, an additional thread is added to record the time cost of inserting rules for each algorithm instance. This constructs a rule lookup cost model for each algorithm instance, which serves as the data basis for measuring the performance and complementarity of different algorithm instances.

[0082] Step S2: Use a genetic algorithm to generate the best combination of different instances of the algorithm. After finding the algorithm instances that meet the requirements, build an algorithm instance repository.

[0083] Specifically, in step S2:

[0084] After determining the cost quantification model based on given rules, an efficient optimal algorithm instance combination method based on genetic algorithm is proposed:

[0085] Reduce the number of candidate algorithm instances: simplify the permutation problem into a combinatorial problem; filter algorithm instances whose average rule search cost is higher than a preset standard; filter algorithm instances whose memory usage is higher than a preset standard.

[0086] Find a combination of algorithm instances that minimizes the overall rule cost, which can be abstracted into the following formula:

[0087]

[0088] Where Ω represents the total space of possible combinations, τ represents a specific combination, N is the number of rules, and cost[i][j] refers to the algorithm instance AI. i The search cost of rule j in the algorithm; AI is an abbreviation for Algorithm Instance.

[0089] Genetic algorithms are used to process rule sets and combinations of algorithm instances. In genetic algorithms, the fitness of each feasible solution is evaluated based on multiple metrics. The overall performance of each algorithm instance is represented by its average rule cost. i The average rule cost is stored in the array avgCost[i], and the reciprocal of avgCost[i] is used as the AI. iThe fitness value, fitness[i], is shown in Formula (2). A higher fitness value corresponds to a lower average cost, and algorithm instances with higher fitness values ​​have a greater chance of being selected. Formula (3) is used to calculate the selection of AI. i The probability of probability[i] is used to sort the candidate algorithm instance set in descending order based on the fitness value. Sort the algorithm instances in the table:

[0090]

[0091]

[0092] The two-dimensional array `cost` stores the rule search cost corresponding to each algorithm instance, where `cost[i][j]` represents the AI. i The search cost of rule j is calculated, and the fitness and selection probability of each algorithm instance are calculated. The set of valid candidate algorithm instances is represented by N, which represents the total number of rules in the rule set. Each iteration round includes: evolution of candidate algorithm instances; updating of the algorithm instance library and acquisition of new combination schemes; and evaluation to determine whether the termination condition is met.

[0093] Specifically, the evolution process of candidate algorithm instances includes replication and crossover: A roulette wheel algorithm is used to determine the algorithm instance selected in each round. First, a random number from 0 to 1 is generated and compared sequentially with the cumulative probability of each algorithm instance. If the probability exceeds the random number, the corresponding algorithm instance is selected. The selection process is proportional to the probability defined in formula (3); Crossover is performed: The two selected algorithm instances are denoted as AI. j and AI k The two algorithms are combined to form a new AI′, where the cost of each rule in AI′ corresponds to the smaller of the two algorithm instances; the greater the complementarity between the different instances of the algorithm, the lower the cost of the newly obtained algorithm instance AI′. The fitness of AI′ is calculated and denoted as curFitness.

[0094] Updates to the algorithm instance library and acquisition of combinatorial schemes: If the current fitness value curFitness is greater than the previous fitness value preFitness, it means that the current crossover round has produced the best result to date, and the new AI' is considered from the set. Removed previous AI j and AI k Insert AI′, calculate the selection probability of AI′ and record its algorithm instance combination scheme;

[0095] Determine if the algorithm meets the termination conditions: The algorithm includes two termination conditions. First, when the number of iterations reaches a threshold ρ, the algorithm stops executing, setting an upper limit on the number of iterations. Second, when the difference between the fitness values ​​generated by two consecutive iterations is less than a threshold ε, the algorithm terminates. The algorithm ends execution when either of these conditions is met, obtaining the desired outcome. The combination of algorithm instances with the highest fitness is denoted as τ. * And return;

[0096] The algorithm has a time complexity of O(Nρ), where N and ρ represent the number of rules in the rule set and the number of iterations, respectively. The algorithm consists of three parts, which are executed in each iteration. First, the algorithm iterates through all rules to form AI′, with a complexity of O(N). Second, the algorithm updates AI by deleting and inserting rules. The complexity of the operation is expressed as The third part involves checking the termination condition, implemented with O(1) complexity; the algorithm iterates at most ρ rounds, with a total complexity of O(1). Given Furthermore, experimental results show that the algorithm typically converges in less than 5 iterations, thus the overall complexity is approximately O(N).

[0097] Step S3: Construct a data packet classification cost prediction model and select a suitable algorithm instance from the constructed algorithm instance library to complete the data packet classification operation.

[0098] Specifically, in step S3:

[0099] A linear regression prediction model is proposed, which selects the optimal algorithm instance for each incoming data packet while considering load balancing.

[0100] An incremental training strategy is used to address the cold start problem; initially, an algorithm instance is randomly selected from the library for packet classification, and each packet P is associated with a triplet.<P,AI,t> The related features are: AI represents the selected algorithm instance, t represents the search time; a preset number of samples are accumulated, the prediction model selects the algorithm instance for data packet classification, the prediction model performs incremental updates to adapt to distribution drift and improve the overall robustness of the solution;

[0101] The linear regression model is represented as follows:

[0102] y=α0+α1x1+α2x2+α3x3 (4)

[0103] Where y represents the estimated message lookup time, x i Indicates the extracted bytes, α i This represents the parameters that the model needs to learn;

[0104] The algorithm instances whose workloads are in a preset saturation state are dynamically masked. A queue is established for each algorithm instance in the repository. When there are waiting data packets in the queue, the corresponding algorithm instance is temporarily marked as unavailable, and the remaining unsaturated algorithm instances are available for selection.

[0105] Example 2:

[0106] Example 2 is a preferred example of Example 1, and is used to illustrate the present invention in more detail.

[0107] The present invention also provides a parallel packet classification system based on ensemble learning. The parallel packet classification system based on ensemble learning can be implemented by executing the process steps of the parallel packet classification method based on ensemble learning. That is, those skilled in the art can understand the parallel packet classification method based on ensemble learning as a preferred embodiment of the parallel packet classification system based on ensemble learning.

[0108] According to the present invention, a parallel packet classification system based on ensemble learning includes:

[0109] Module M1: Establishes a rule-based search cost model and quantifies the complementarity between different instances of the algorithm;

[0110] Specifically, in module M1:

[0111] A search cost quantification model based on rules is proposed to measure the complementarity between different instances of the algorithm. In order to determine complementary algorithm instances, from the perspective of rules, when considering a specific algorithm instance, the subset of rules that generate the highest search cost is determined. For complementary algorithm instances, their respective high-cost rule subsets have the least overlap.

[0112] The spatial partitioning step is represented as the internal nodes of a tree, and the partitioned subspaces are represented as leaf nodes. The cost of rule lookup is modeled from the perspective of rule insertion. The depth of reaching a leaf node in the tree structure represents the cost required to find a rule. During the construction process, the algorithm determines the optimal parameter configuration from the search space. Each parameter configuration corresponds to an algorithm instance. In the process of searching for the optimal parameter configuration, an additional thread is added to record the time cost of inserting rules for each algorithm instance. This constructs a rule lookup cost model for each algorithm instance, which serves as the data basis for measuring the performance and complementarity of different algorithm instances.

[0113] Module M2: Uses a genetic algorithm to generate the best combination of different instances of the algorithm, and builds an algorithm instance repository after finding algorithm instances that meet the requirements;

[0114] Specifically, in module M2:

[0115] After determining the cost quantification model based on given rules, an efficient optimal algorithm instance combination method based on genetic algorithm is proposed:

[0116] Reduce the number of candidate algorithm instances: simplify the permutation problem into a combinatorial problem; filter algorithm instances whose average rule search cost is higher than a preset standard; filter algorithm instances whose memory usage is higher than a preset standard.

[0117] Find a combination of algorithm instances that minimizes the overall rule cost, which can be abstracted into the following formula:

[0118]

[0119] Where Ω represents the total space of possible combinations, τ represents a specific combination, N is the number of rules, and cost[i][j] refers to the algorithm instance AI. i The search cost of rule j in the algorithm; AI is an abbreviation for Algorithm Instance.

[0120] Genetic algorithms are used to process rule sets and combinations of algorithm instances. In genetic algorithms, the fitness of each feasible solution is evaluated based on multiple metrics. The overall performance of each algorithm instance is represented by its average rule cost. i The average rule cost is stored in the array avgCost[i], and the reciprocal of avgCost[i] is used as the AI. i The fitness value, fitness[i], is shown in Formula (2). A higher fitness value corresponds to a lower average cost, and algorithm instances with higher fitness values ​​have a greater chance of being selected. Formula (3) is used to calculate the selection of AI. i The probability of probability[i] is used to sort the candidate algorithm instance set in descending order based on the fitness value. Sort the algorithm instances in the table:

[0121]

[0122]

[0123] The two-dimensional array `cost` stores the rule search cost corresponding to each algorithm instance, where `cost[i][j]` represents the AI. i The search cost of rule j is calculated, and the fitness and selection probability of each algorithm instance are calculated. The set of valid candidate algorithm instances is represented by N, which represents the total number of rules in the rule set. Each iteration round includes: evolution of candidate algorithm instances; updating of the algorithm instance library and acquisition of new combination schemes; and evaluation to determine whether the termination condition is met.

[0124] Specifically, the evolution process of candidate algorithm instances includes replication and crossover: A roulette wheel algorithm is used to determine the algorithm instance selected in each round. First, a random number from 0 to 1 is generated and compared sequentially with the cumulative probability of each algorithm instance. If the probability exceeds the random number, the corresponding algorithm instance is selected. The selection process is proportional to the probability defined in formula (3); Crossover is performed: The two selected algorithm instances are denoted as AI. j and AI k The two algorithms are combined to form a new AI′, where the cost of each rule in AI′ corresponds to the smaller of the two algorithm instances; the greater the complementarity between the different instances of the algorithm, the lower the cost of the newly obtained algorithm instance AI′. The fitness of AI′ is calculated and denoted as curFitness.

[0125] Updates to the algorithm instance library and acquisition of combinatorial schemes: If the current fitness value curFitness is greater than the previous fitness value preFitness, it means that the current crossover round has produced the best result to date, and the new AI' is considered from the set. Removed previous AI j and AI k Insert AI′, calculate the selection probability of AI′ and record its algorithm instance combination scheme;

[0126] Determine if the algorithm meets the termination conditions: The algorithm includes two termination conditions. First, when the number of iterations reaches a threshold ρ, the algorithm stops executing, setting an upper limit on the number of iterations. Second, when the difference between the fitness values ​​generated by two consecutive iterations is less than a threshold ε, the algorithm terminates. The algorithm ends execution when either of these conditions is met, obtaining the desired outcome. The combination of algorithm instances with the highest fitness is denoted as τ. * And return;

[0127] The algorithm has a time complexity of O(Nρ), where N and ρ represent the number of rules in the rule set and the number of iterations, respectively. The algorithm consists of three parts, which are executed in each iteration. First, the algorithm iterates through all rules to form AI′, with a complexity of O(N). Second, the algorithm updates AI by deleting and inserting rules. The complexity of the operation is expressed as The third part involves checking the termination condition, implemented with O(1) complexity; the algorithm iterates at most ρ rounds, with a total complexity of O(1). Given Furthermore, experimental results show that the algorithm typically converges in less than 5 iterations, thus the overall complexity is approximately O(N).

[0128] Module M3: Constructs a data packet classification cost prediction model and selects appropriate algorithm instances from the constructed algorithm instance library to complete the data packet classification operation.

[0129] Specifically, in module M3:

[0130] A linear regression prediction model is proposed, which selects the optimal algorithm instance for each incoming data packet while considering load balancing.

[0131] An incremental training strategy is used to address the cold start problem; initially, an algorithm instance is randomly selected from the library for packet classification, and each packet P is associated with a triplet.<P,AI,t> The related features are: AI represents the selected algorithm instance, t represents the search time; a preset number of samples are accumulated, the prediction model selects the algorithm instance for data packet classification, the prediction model performs incremental updates to adapt to distribution drift and improve the overall robustness of the solution;

[0132] The linear regression model is represented as follows:

[0133] y=α0+α1x1+α2x2+α3x3 (4)

[0134] Where y represents the estimated message lookup time, x i Indicates the extracted bytes, α i This represents the parameters that the model needs to learn;

[0135] The algorithm instances whose workloads are in a preset saturation state are dynamically masked. A queue is established for each algorithm instance in the repository. When there are waiting data packets in the queue, the corresponding algorithm instance is temporarily marked as unavailable, and the remaining unsaturated algorithm instances are available for selection.

[0136] Example 3:

[0137] Example 3 is a preferred example of Example 1, and is used to illustrate the present invention in more detail.

[0138] The technical solution of this invention: This patent mainly solves the following three key problems: how to quantify the complementarity between different instances of the algorithm, how to generate the best combination of algorithm instances, and how to select a suitable algorithm instance for each data packet.

[0139] First, we establish a general rule search cost model to quantify the complementarity between different instances of the algorithm. This model measures the difference in the worst-case longest rule subsets and the overall performance difference between different algorithm instances. Second, we use a genetic algorithm to generate optimal combinations of different algorithm instances. By leveraging the evolutionary search capability of the genetic algorithm, we can efficiently explore and identify the most effective combinations of algorithm instances, thereby maximizing performance stability and throughput. Third, we construct a packet classification cost prediction model to select suitable algorithm instances from a constructed algorithm instance library to perform packet classification operations. This selection process considers load balancing to ensure that the selected algorithm instances can efficiently process packets while maintaining optimal throughput.

[0140] 1. Analysis of existing methods:

[0141] Existing packet classification methods can be broadly categorized into four types. The first type is hardware-based methods. These methods utilize dedicated hardware (typically Tri-State Content-Addressable Memory, TCAM) to accelerate lookup operations, demonstrating excellent performance in achieving linear-speed classification. The second type is decision tree-based algorithms. These methods use tree structures to organize rules and employ spatial partitioning methods to distribute rules into different subspaces. These techniques enable fast lookups. The third type is partitioning algorithms. These methods divide the rule set into multiple subsets based on rule feature analysis and use hash tables to organize these subsets. By assigning each rule to a unique subset, rule duplication is eliminated, and hash tables support efficient updates. The fourth type is learning-based indexing algorithms. These methods combine classic algorithm structures with machine learning, or transform the rule set into a neural network model, treating it as a model prediction problem, achieving good results.

[0142] 2. Overall Framework

[0143] Figure 1 This paper presents the overall framework of a packet parallel classification method based on ensemble learning, comprising three modules. First, a general quantitative model is established to measure search cost from a rule-based perspective, which forms the basis for selecting complementary algorithm instances. Second, a genetic algorithm-based method is proposed to search for the optimal combination of algorithm instances. After finding suitable algorithm instances, an algorithm instance repository can be constructed. Third, a lightweight prediction model is built to select a suitable algorithm instance for each incoming packet.

[0144] 3. Rule-based search cost quantification model

[0145] This patent first proposes a general quantitative model for search cost based on rules to measure the complementarity between different instances of an algorithm. Long tails and worst-case scenarios in algorithm performance are mainly due to certain rules existing in large buckets or long search paths. Therefore, to determine complementary algorithm instances, a rule-based approach can be taken. Generally, when considering a specific algorithm instance, it is necessary to identify the subset of rules that generate the highest search cost. For complementary algorithm instances, their respective high-cost rule subsets should overlap as little as possible.

[0146] By analyzing the rule organization of packet classification algorithms, we can observe that they can typically be abstracted as a tree structure. Specifically, the spatial partitioning step can be represented as the internal nodes of the tree, while the partitioned subspaces are represented as leaf nodes. We further observe that finding a rule in the tree usually requires following the same path as the rule insertion during construction. Therefore, the cost of rule lookup can be modeled from the perspective of rule insertion. Generally, the depth of reaching a leaf node in the tree structure can represent the cost required to find a rule. The deeper the tree, the higher the cost of locating the rule. In addition, most algorithms need to determine the optimal parameter configuration from the search space during construction, with each parameter configuration corresponding to an algorithm instance. Therefore, during the search for the optimal parameter configuration, we can add an additional thread to record the time cost of inserting rules for each algorithm instance. This allows us to construct a rule lookup cost model for each algorithm instance, serving as a data foundation for measuring the performance and complementarity of different algorithm instances.

[0147] 4. Optimal Algorithm Instance Combination Generation Scheme

[0148] This patent proposes an efficient optimal algorithm instance combination method based on genetic algorithms after finding a cost quantification model with given rules.

[0149] This method first reduces the complexity of this combinatorial optimization problem by decreasing the number of candidate algorithm instances. This is mainly based on three aspects: First, some bag classification methods require consideration of different parameter permutations. Algorithm instances determined by different permutations of the same parameter settings have high similarity. Therefore, the permutation problem can be simplified to a combinatorial problem. Second, algorithm instances with high average rule search costs contribute little to performance optimization in the worst case and perform poorly in load balancing, so they can be safely filtered out. Third, algorithm instances with high memory usage typically contribute little to performance improvement but have a significant impact on memory overhead, so they should also be filtered out. Based on these three points, we can significantly reduce the search space of candidate algorithm instances. Since the ultimate goal is to find a combination scheme of algorithm instances that minimizes the overall rule cost, the problem can be abstracted into the following formula:

[0150]

[0151] Where Ω represents the total space of possible combinations, τ represents a specific combination, N is the number of rules, and cost[i][j] refers to the algorithm instance AI. i The search cost of rule j. AI is an abbreviation for Algorithm Instance. Since the rule set is almost 256k or even larger, and the problem size is a combination of different candidate algorithm instances, these two factors lead to increased complexity. The evolutionary process in genetic algorithms, including replication and crossover, aligns with the need to preserve the optimal algorithm instance and find complementary algorithm instances; simultaneously, by limiting the search space, genetic algorithms can quickly converge to a solution, effectively reducing algorithm complexity. Therefore, this patent uses genetic algorithms to efficiently handle large-scale rule sets and a large number of algorithm instance combinations.

[0152] In genetic algorithms, the fitness of each feasible solution can be evaluated based on multiple metrics. In this problem context, the overall performance of each algorithm instance can be represented by its average rule cost. It is worth noting that AI... i The average rule cost is stored in the array avgCost[i]. Therefore, we take the reciprocal of avgCost[i] as the AI. i The fitness value, fitness[i], is shown in Equation (2). It can be seen that a higher fitness value corresponds to a lower average cost. Furthermore, during the evolutionary process, the fitness value plays a crucial role in determining the probability of selecting an algorithm instance. Algorithm instances with higher fitness values ​​have a greater chance of being selected. Specifically, we use Equation (3) to calculate the selection of AI. i The probability of probability[i]. Then, we sort the candidate algorithm instance set in descending order based on their fitness values. Sort the algorithm instances in the algorithm, which helps in the subsequent selection process.

[0153]

[0154]

[0155] The flowchart of the algorithm for generating the optimal combination of algorithm instances based on the genetic algorithm is as follows: Figure 2 As shown. The two-dimensional array `cost` is used to store the rule search cost corresponding to each algorithm instance. More specifically, `cost[i][j]` represents the AI... i The search cost of rule j. Before iteration, we first calculate the fitness and selection probability of each algorithm instance. The set of valid candidate algorithm instances is represented by N, and the total number of rules in the rule set is represented by N. Each iteration round consists of three distinct components: 1) evolution of candidate algorithm instances; 2) updating the algorithm instance library and obtaining new combination schemes; and 3) evaluation to determine whether the termination condition is met.

[0156] The first part covers the evolutionary process of candidate algorithm instances, including replication and crossover. Initially, a roulette wheel algorithm is used to determine the algorithm instance selected in each round. This strategy considers both fitness and randomness. Specifically, a random number from 0 to 1 is generated first, and then compared with the cumulative probability of each algorithm instance in turn. Once the probability exceeds the random number, the corresponding algorithm instance is selected. The selection process is proportional to the probability defined in formula (3). Crossover is then performed: the two selected algorithm instances are denoted as AI. j and AI k These two instances of the algorithm are combined to form a new AI′, where the cost of each rule in AI′ corresponds to the smaller of the two algorithm instances. The remaining algorithm instances are temporarily retained, awaiting further selection. Furthermore, this process also considers the complementarity between different instances of the quantification algorithm. The greater the complementarity, the lower the cost of the newly obtained algorithm instance AI′. Finally, the fitness of AI′, denoted as curFitness, is calculated.

[0157] The second part involves updating the algorithm instance library and obtaining combinatorial schemes. If the current fitness value (curFitness) is greater than the previous fitness value (preFitness), it means that the current crossover round has produced the best result so far, and the new AI' should be considered. Therefore, we start from the set... Removed previous AI j and AI k We then insert AI′. Simultaneously, we calculate the selection probability of AI′ and record its algorithmic instance combinations to ensure its application in subsequent iterations.

[0158] The third part determines whether the algorithm meets the termination conditions. The algorithm has two termination conditions. First, the algorithm stops executing when the number of iterations reaches a threshold ρ. To strike a balance between accuracy and efficiency, it's necessary to set an upper limit on the number of iterations. Second, the algorithm terminates when the difference between the fitness values ​​produced by two consecutive iterations is less than a threshold ε. A small difference means that subsequent iterations will only bring minor improvements, and the current combination has already achieved a relatively ideal result, so the iteration can exit. The algorithm ends execution when either of the above conditions is met. Finally, we obtain... The combination of algorithm instances with the highest fitness is denoted as τ. * And return.

[0159] The algorithm has a time complexity of O(Nρ), where N and ρ represent the number of rules in the rule set and the number of iterations, respectively. The algorithm consists of three parts, which are executed in each iteration. First, the algorithm iterates through all rules to form AI′, with a complexity of O(N). Second, the algorithm updates AI by deleting and inserting rules. The complexity of these operations can be expressed as The third part involves checking the termination condition, which can be implemented with O(1) complexity. The algorithm iterates at most ρ rounds, therefore the total complexity is O(1). Given Furthermore, experimental results show that the algorithm typically converges in less than 5 iterations, so the overall complexity can be approximated as O(N).

[0160] 5. Algorithm instance selection scheme based on prediction

[0161] This patent proposes a linear regression prediction model based on a given library of algorithm instances. This model can select the optimal algorithm instance for each incoming data packet while considering load balancing. The inference process of the prediction model should be executed as quickly as possible to prevent an increase in overall lookup time. Furthermore, considering factors such as memory usage, prediction accuracy, and update speed, a simple linear regression model was chosen. The advantage of this model is that it consumes very little memory and time.

[0162] This scheme employs an incremental training strategy to effectively address the cold start problem. Initially, an algorithm instance is randomly selected from the library for packet classification. Each packet P is associated with a triplet.<P,AI,t> The algorithm is associated with AI, where AI represents the selected algorithm instance and t represents the lookup time. These samples are then used to train the prediction model. Once a sufficient number of samples have been accumulated, achieving a high level of prediction accuracy, the prediction model is able to select suitable algorithm instances for packet classification. Furthermore, the prediction model can be incrementally updated to adapt to distribution drift and improve the overall robustness of the scheme.

[0163] This patent analyzed byte-level feature correlations and found that packets with longer lookup times exhibited a certain degree of clustering in the high bytes of both SIP and DIP. This observation prompted us to choose fewer bytes as input features for the linear regression model to strike a balance between accuracy and overhead. The linear regression model is represented as follows:

[0164] y=α0+α1x1+α2x2+α3x3#(4)

[0165] Where y represents the estimated message lookup time, x i Indicates the extracted bytes, α i This represents the parameters that the model needs to learn.

[0166] Meanwhile, to achieve load balancing, this patent dynamically masks algorithm instances whose workload is saturated. Specifically, if there is a concentrated packet flow in the data packets, it may over-select the algorithm instance with the shortest estimated lookup time, leading to packet congestion for that algorithm instance. In implementation, a queue is established for each algorithm instance in the repository. When there are waiting packets in the queue, the corresponding algorithm instance is temporarily marked as unavailable. The remaining unsaturated algorithm instances are then available for selection. Given that the overall performance of algorithm instances in the instance library is similar, this approach not only increases throughput but also alleviates the long-tail problem.

[0167] Table 1: Description of the meanings of commonly used symbols

[0168]

[0169] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0170] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A parallel data packet classification method based on ensemble learning, characterized in that, include: Step S1: Establish a rule-based search cost model and quantify the complementarity between different instances of the algorithm; Step S2: Use a genetic algorithm to generate the best combination of different instances of the algorithm. After finding the algorithm instances that meet the requirements, build an algorithm instance repository. In step S2: After determining the cost quantification model based on given rules, an efficient optimal algorithm instance combination method based on genetic algorithm is proposed: Reduce the number of candidate algorithm instances: simplify the permutation problem into a combinatorial problem; filter algorithm instances whose average rule search cost is higher than a preset standard; filter algorithm instances whose memory usage is higher than a preset standard. Find a combination of algorithm instances that minimizes the overall rule cost, which can be abstracted into the following formula: in, Represents the overall combination scheme space. This represents a specific combination scheme. It is the number of rules, and Refers to algorithm instances Chinese rules Search costs It is an abbreviation for Algorithm Instance; Genetic algorithms are used to process combinations of rule sets and algorithm instances. In genetic algorithms, the fitness of each feasible solution is evaluated based on multiple metrics, and the overall performance of each algorithm instance is represented by its average rule cost. The average rule cost is stored in an array In China, the following is adopted The reciprocal of fitness value As shown in formula (2), use formula (3) to calculate the selection. probability The candidate algorithm instance set is sorted in descending order based on fitness value. Sort the algorithm instances in the table: Two-dimensional array Store the rule search cost for each algorithm instance. express Chinese rules The search cost is calculated, and the fitness and selection probability of each algorithm instance are determined. This represents the set of valid candidate algorithm instances. This represents the total number of rules in the rule set; each iteration round includes: evolution of candidate algorithm instances; updating the algorithm instance library and obtaining new combination schemes; evaluation to determine whether the termination condition is met; Step S3: Construct a data packet classification cost prediction model, and select a suitable algorithm instance from the constructed algorithm instance library to complete the data packet classification operation; In step S3: A linear regression prediction model is proposed, which selects the optimal algorithm instance for each incoming data packet while considering load balancing. An incremental training strategy is used to address the cold start problem; initially, an algorithm instance is randomly selected from the library for packet classification, and each packet... Both are associated with a triplet Related, among which Indicates the selected algorithm instance, Indicates the search time; a preset number of samples have been accumulated, the prediction model selects algorithm instances for data packet classification, the prediction model performs incremental updates to adapt to distribution drift and improve the overall robustness of the solution; The linear regression model is represented as follows: in, This represents the estimated message lookup time. This indicates the extracted bytes. This represents the parameters that the model needs to learn; The algorithm instances whose workloads are in a preset saturation state are dynamically masked. A queue is established for each algorithm instance in the repository. When there are waiting data packets in the queue, the corresponding algorithm instance is temporarily marked as unavailable, and the remaining unsaturated algorithm instances are available for selection.

2. The parallel packet classification method based on ensemble learning according to claim 1, characterized in that, In step S1: A search cost quantification model based on rules is proposed to measure the complementarity between different instances of the algorithm. In order to determine complementary algorithm instances, from the perspective of rules, when considering a specific algorithm instance, the subset of rules that generate the highest search cost is determined. For complementary algorithm instances, their respective high-cost rule subsets have the least overlap. The spatial partitioning step is represented as the internal nodes of a tree, and the partitioned subspaces are represented as leaf nodes. The cost of rule lookup is modeled from the perspective of rule insertion. The depth of reaching a leaf node in the tree structure represents the cost required to find a rule. During the construction process, the algorithm determines the optimal parameter configuration from the search space. Each parameter configuration corresponds to an algorithm instance. In the process of searching for the optimal parameter configuration, an additional thread is added to record the time cost of inserting rules for each algorithm instance. This constructs a rule lookup cost model for each algorithm instance, which serves as the data basis for measuring the performance and complementarity of different algorithm instances.

3. The parallel packet classification method based on ensemble learning according to claim 1, characterized in that: The evolution process of candidate algorithm instances includes replication and crossover: A roulette wheel algorithm is used to determine the algorithm instance selected in each round. First, a random number from 0 to 1 is generated and compared with the cumulative probability of each algorithm instance. If the probability exceeds the random number, the corresponding algorithm instance is selected. The selection process is proportional to the probability defined in formula (3); Crossover is performed: The two selected algorithm instances are represented as follows: and , combined to form a new ,in The cost of each rule corresponds to the smaller of the two algorithm instances; the greater the complementarity between different instances of the algorithm, the better the newly obtained algorithm instance. The lower the cost, the better the calculation. fitness, denoted as ; Updating the algorithm instance library and obtaining combinatorial schemes: if the current fitness value is satisfied. greater than the previous fitness value This indicates that the current crossover round has produced the best result to date, and the new... Taken into consideration, from the set Removed from the previous and , and insert ,calculate The selection probability is recorded, and the algorithm instance combination scheme is recorded. Determine if the algorithm meets the termination condition: The algorithm has two termination conditions, one when the number of iterations reaches a threshold. When the algorithm stops executing, an upper limit is set on the number of iterations. When the difference between the fitness values ​​generated by two consecutive iterations is less than a threshold The algorithm terminates when any of the above conditions are met, and the algorithm obtains the desired result. The combination of algorithm instances with the highest fitness is denoted as . And return; The time complexity of the algorithm is ,in and These represent the number of rules in the rule set and the number of iterations, respectively. The algorithm consists of three parts, which are executed in each iteration. First, the algorithm iterates through all the rules to form... Its complexity is ; Secondly, the algorithm uses deletion and insertion. renew The complexity of the operation is expressed as Part Three involves checking the termination conditions, through... The algorithm achieves a time complexity of [missing information]; the algorithm has a maximum number of iterations. The total complexity is (The rest of the text is incomplete and cannot be translated.) Given Furthermore, experimental results show that the algorithm typically converges in less than 5 iterations, therefore the overall complexity is approximately O(n). .

4. A parallel data packet classification system based on ensemble learning, characterized in that, include: Module M1: Establishes a rule-based search cost model and quantifies the complementarity between different instances of the algorithm; Module M2: Uses a genetic algorithm to generate the best combination of different instances of the algorithm, and builds an algorithm instance repository after finding algorithm instances that meet the requirements; In module M2: After determining the cost quantification model based on given rules, an efficient optimal algorithm instance combination method based on genetic algorithm is proposed: Reduce the number of candidate algorithm instances: simplify the permutation problem into a combinatorial problem; filter algorithm instances whose average rule search cost is higher than a preset standard; filter algorithm instances whose memory usage is higher than a preset standard. Find a combination of algorithm instances that minimizes the overall rule cost, which can be abstracted into the following formula: in, Represents the overall combination scheme space. This represents a specific combination scheme. It is the number of rules, and Refers to algorithm instances Chinese rules Search costs It is an abbreviation for Algorithm Instance; Genetic algorithms are used to process combinations of rule sets and algorithm instances. In genetic algorithms, the fitness of each feasible solution is evaluated based on multiple metrics, and the overall performance of each algorithm instance is represented by its average rule cost. The average rule cost is stored in an array In China, the following is adopted The reciprocal of fitness value As shown in formula (2), use formula (3) to calculate the selection. probability The candidate algorithm instance set is sorted in descending order based on fitness value. Sort the algorithm instances in the table: Two-dimensional array Store the rule search cost for each algorithm instance. express Chinese rules The search cost is calculated, and the fitness and selection probability of each algorithm instance are determined. This represents the set of valid candidate algorithm instances. This represents the total number of rules in the rule set; each iteration round includes: evolution of candidate algorithm instances; updating the algorithm instance library and obtaining new combination schemes; evaluation to determine whether the termination condition is met; Module M3: Constructs a packet classification cost prediction model and selects appropriate algorithm instances from the constructed algorithm instance library to complete the packet classification operation; In module M3: A linear regression prediction model is proposed, which selects the optimal algorithm instance for each incoming data packet while considering load balancing. An incremental training strategy is used to address the cold start problem; initially, an algorithm instance is randomly selected from the library for packet classification, and each packet... Both are associated with a triplet Related, among which Indicates the selected algorithm instance, Indicates the search time; a preset number of samples have been accumulated, the prediction model selects algorithm instances for data packet classification, the prediction model performs incremental updates to adapt to distribution drift and improve the overall robustness of the solution; The linear regression model is represented as follows: in, This represents the estimated message lookup time. This indicates the extracted bytes. This represents the parameters that the model needs to learn; The algorithm instances whose workloads are in a preset saturation state are dynamically masked. A queue is established for each algorithm instance in the repository. When there are waiting data packets in the queue, the corresponding algorithm instance is temporarily marked as unavailable, and the remaining unsaturated algorithm instances are available for selection.

5. The parallel data packet classification system based on ensemble learning according to claim 4, characterized in that, In module M1: A search cost quantification model based on rules is proposed to measure the complementarity between different instances of the algorithm. In order to determine complementary algorithm instances, from the perspective of rules, when considering a specific algorithm instance, the subset of rules that generate the highest search cost is determined. For complementary algorithm instances, their respective high-cost rule subsets have the least overlap. The spatial partitioning step is represented as the internal nodes of a tree, and the partitioned subspaces are represented as leaf nodes. The cost of rule lookup is modeled from the perspective of rule insertion. The depth of reaching a leaf node in the tree structure represents the cost required to find a rule. During the construction process, the algorithm determines the optimal parameter configuration from the search space. Each parameter configuration corresponds to an algorithm instance. In the process of searching for the optimal parameter configuration, an additional thread is added to record the time cost of inserting rules for each algorithm instance. This constructs a rule lookup cost model for each algorithm instance, which serves as the data basis for measuring the performance and complementarity of different algorithm instances.

6. The parallel data packet classification system based on ensemble learning according to claim 4, characterized in that: The evolution process of candidate algorithm instances includes replication and crossover: A roulette wheel algorithm is used to determine the algorithm instance selected in each round. First, a random number from 0 to 1 is generated and compared with the cumulative probability of each algorithm instance. If the probability exceeds the random number, the corresponding algorithm instance is selected. The selection process is proportional to the probability defined in formula (3); Crossover is performed: The two selected algorithm instances are represented as follows: and , combined to form a new ,in The cost of each rule corresponds to the smaller of the two algorithm instances; the greater the complementarity between different instances of the algorithm, the better the newly obtained algorithm instance. The lower the cost, the better the calculation. fitness, denoted as ; Updating the algorithm instance library and obtaining combinatorial schemes: if the current fitness value is satisfied. greater than the previous fitness value This indicates that the current crossover round has produced the best result to date, and the new... Taken into consideration, from the set Removed from the previous and , and insert ,calculate The selection probability is recorded, and the algorithm instance combination scheme is recorded. Determine if the algorithm meets the termination condition: The algorithm has two termination conditions, one when the number of iterations reaches a threshold. When the algorithm stops executing, an upper limit is set on the number of iterations. When the difference between the fitness values ​​generated by two consecutive iterations is less than a threshold The algorithm terminates when any of the above conditions are met, and the algorithm obtains the desired result. The combination of algorithm instances with the highest fitness is denoted as . And return; The time complexity of the algorithm is ,in and These represent the number of rules in the rule set and the number of iterations, respectively. The algorithm consists of three parts, which are executed in each iteration. First, the algorithm iterates through all the rules to form... Its complexity is Secondly, the algorithm uses deletion and insertion... renew The complexity of the operation is expressed as Part Three involves checking the termination conditions, through... The algorithm achieves a time complexity of [missing information]; the algorithm has a maximum number of iterations. The total complexity is (The rest of the text is incomplete and cannot be translated.) Given Furthermore, experimental results show that the algorithm typically converges in less than 5 iterations, therefore the overall complexity is approximately O(n). .

Images