Dynamic network flow control method and device, equipment and storage medium

Abstract

The invention discloses a dynamic network flow control method, which comprises the following steps of: preprocessing acquired network state information, and performing feature extraction on the preprocessed network state information by utilizing a feature function to obtain a feature vector set; on the basis of the feature vector set, carrying out iterative computation by adopting an ant colony algorithm to obtain an optimal routing path; and inputting the optimal routing path into a reinforcement learning model, outputting a flow control strategy, and performing network flow control according to the flow control strategy, according to the method, the ant colony algorithm and reinforcement learning are combined, the precision and stability of network flow prediction can be improved, balanced distribution and efficient utilization of network flow are achieved, and therefore the overall performance of a network is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network flow control, and in particular to a dynamic network flow control method, device, equipment and storage medium. Background Art

[0002] With the continuous enrichment of network applications, network traffic types are becoming increasingly diverse. In addition to traditional applications such as Web browsing and file transfer, a large number of high-bandwidth, low-latency applications based on video, audio, real-time communication, etc. have emerged. These applications have different requirements for network traffic and require dynamic network traffic management to reasonably allocate bandwidth resources. At the same time, network traffic often has a bursty characteristic, that is, the traffic will suddenly increase in certain time periods, while it is relatively stable in other time periods. This bursty traffic characteristic requires the network traffic management system to be able to perceive traffic changes in real time and quickly adjust the traffic management strategy to adapt to changes in network status.

[0003] Some existing technologies use prediction models based on statistics and machine learning, such as autoregressive models and support vector machines. When processing complex and changeable network traffic data, it is difficult to capture all factors that affect traffic, resulting in limited prediction accuracy. In order to improve the prediction accuracy, some advanced prediction models such as deep learning models are used, but they have high complexity, resulting in huge computational complexity in the model training and prediction process, and high requirements for hardware resources. Changes in the network environment may also cause the prediction model to fail or the prediction results to deviate from the actual situation, making it difficult to achieve balanced distribution and efficient utilization of network traffic. Summary of the invention

[0004] The embodiment of the present invention provides a dynamic network traffic control method, which combines the ant colony algorithm with reinforcement learning, can improve the accuracy and stability of network traffic prediction, achieve balanced distribution and efficient utilization of network traffic, and thus improve the overall performance of the network.

[0005] In a first aspect, an embodiment of the present invention provides a dynamic network traffic control method, comprising:

[0006] Preprocessing the collected network status information, extracting features from the preprocessed network status information using a feature function, and obtaining a feature vector set;

[0007] Based on the feature vector set, an ant colony algorithm is used for iterative calculation to obtain an optimal routing path;

[0008] The optimal routing path is input into a reinforcement learning model, a flow control strategy is output, and network flow control is performed according to the flow control strategy.

[0009] Furthermore, the feature function is used to extract features from the preprocessed network status information to obtain a feature vector set, including:

[0010] The data packet size feature value extraction function, data packet speed feature value extraction function and data packet direction feature value extraction function are respectively used to extract features of the pre-processed network status information to obtain the data packet size feature value, data packet speed feature value and data packet direction feature value. The mathematical formula is as follows:

[0011]

[0012] Among them, S(t) represents the characteristic value of the packet size, V(t) represents the characteristic value of the packet speed, D(t) represents the characteristic value of the packet direction, and L i Indicates the length of the i-th data packet, R i represents the transmission rate of the ith data packet, O i represents the direction of the ith data packet, and n represents the number of data packets.

[0013] Furthermore, the method of iteratively calculating based on the feature vector set using an ant colony algorithm to obtain an optimal routing path includes:

[0014] Step 1: Initialize the algorithm parameters of the ant colony algorithm; wherein the algorithm parameters include the number of ants, the first maximum number of iterations, the initial value of the pheromone concentration, and the heuristic factor;

[0015] Step 2: Based on the feature vector set, randomly generate an initial path for each ant;

[0016] Step 3: According to the pheromone concentration and heuristic factor of the selected path, the next path is selected for the ant based on the path selection probability formula;

[0017] Step 4: Determine whether the current number of iterations reaches the first maximum number of iterations. If not, repeat the above step 3 until the current number of iterations reaches the first maximum number of iterations. If so, output the best routing path of the ant colony.

[0018] Furthermore, the optimal routing path is input into a reinforcement learning model to output a flow control strategy, including:

[0019] Step 1: input the optimal routing path as an initial strategy into the reinforcement learning model, and initialize the second maximum number of iterations;

[0020] Step 2: Record the current network state, select the next action according to the initial strategy, and execute the action, and calculate the reward obtained after executing the action according to the preset reward function;

[0021] Step 3: Using a reinforcement learning algorithm to update the initial strategy;

[0022] Step 4: Determine whether the current number of iterations has reached the second maximum number of iterations. If not, repeat steps 2 and 3 until the current number of iterations reaches the second maximum number of iterations. If so, perform a performance evaluation on the updated strategy output in step 3 to obtain a first performance evaluation result, and output a final flow control strategy based on the first performance evaluation result.

[0023] Further, outputting a final flow control strategy according to the first performance evaluation result includes:

[0024] If the first evaluation result does not meet the preset requirements, repeat steps 2 and 3 until the first evaluation result meets the preset requirements;

[0025] If the first performance evaluation result meets the preset requirements, the updated strategy is determined as the final flow control strategy and output.

[0026] Furthermore, after performing network traffic control according to the traffic control strategy, the method further includes:

[0027] The network status is monitored in real time, and real-time network status information is obtained, and the flow control strategy is adjusted based on the real-time network status information.

[0028] Furthermore, the method further comprises:

[0029] After adjusting the flow control strategy, performing a performance evaluation on the adjusted flow control strategy to obtain a second performance evaluation result;

[0030] When the second performance evaluation result does not meet the preset requirement, feeding back the adjusted flow control strategy to the reinforcement learning model, so that the reinforcement learning model performs model adjustment according to the second performance evaluation result;

[0031] When the second performance evaluation result meets the preset requirements, network traffic control is performed according to the adjusted traffic control strategy, and the latest network status information and the adjusted traffic control strategy are visualized and returned to relevant technical personnel.

[0032] In a second aspect, an embodiment of the present invention provides a dynamic network traffic control device, comprising:

[0033] The network status feature extraction module is used to pre-process the collected network status information, and use the feature function to extract the features of the pre-processed network status information to obtain a feature vector set;

[0034] An optimal routing path generation module is used to obtain an optimal routing path by iterative calculation using an ant colony algorithm based on the feature vector set;

[0035] The flow control strategy generation module is used to input the optimal routing path into the reinforcement learning model, output the flow control strategy, and perform network flow control according to the flow control strategy.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0037] Memory for storing computer programs;

[0038] A processor, configured to execute the computer program;

[0039] Wherein, when the processor executes the computer program, the dynamic network traffic control method described in any one of the first aspects above is implemented.

[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the dynamic network traffic control method described in any one of the first aspects above is implemented.

[0041] Compared with the prior art, a dynamic network traffic control method provided by an embodiment of the present invention has the beneficial effects of: preprocessing the collected network status information, extracting features of the preprocessed network status information using a characteristic function, and obtaining a feature vector set; based on the feature vector set, an ant colony algorithm is used for iterative calculation to obtain an optimal routing path; the optimal routing path is input into a reinforcement learning model, a traffic control strategy is output, and network traffic control is performed according to the traffic control strategy; the present invention can improve the accuracy and stability of network traffic prediction, and generate and adjust the traffic control strategy according to the real-time changes in the network status, thereby achieving balanced distribution and efficient utilization of network traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical features of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 It is a flow chart of an embodiment of a dynamic network traffic control method provided by the present invention;

[0044] Figure 2 It is a structural schematic diagram of an embodiment of a dynamic network traffic control device provided by the present invention;

[0045] Figure 3 It is a structural schematic diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0049] In a first aspect, an embodiment of the present invention provides a method for dynamic network traffic control. Figure 1 , which is a flow chart of an embodiment of a dynamic network traffic control method provided by the present invention.

[0050] like Figure 1 As shown, the method comprises the following steps:

[0051] S1: preprocessing the collected network status information, extracting features from the preprocessed network status information using a feature function, and obtaining a feature vector set;

[0052] S2: Based on the feature vector set, an ant colony algorithm is used for iterative calculation to obtain an optimal routing path;

[0053] S3: Input the optimal routing path into the reinforcement learning model, output a flow control strategy, and perform network flow control according to the flow control strategy.

[0054] In the specific implementation, we first collect network status information in the network environment, including but not limited to node status, link load, real-time distribution of traffic, network delay and packet loss rate, and preprocess the collected data, including data cleaning, noise removal, outlier and duplicate data removal, to ensure the accuracy and reliability of the data. We use feature functions to extract features from the preprocessed data to obtain key feature vectors that can reflect the network status.

[0055] Based on the feature vector set, an ant colony algorithm is used to perform preliminary planning of network traffic routing, and the planned optimal routing path is output. The reinforcement learning model receives the optimal routing path output by the ant colony algorithm and uses it as the initial strategy. The reinforcement learning algorithm is used to train the intelligent agent. The intelligent agent continuously adjusts the traffic control strategy according to the current network state and reward feedback, and finally outputs the optimized traffic control strategy. The network traffic control is performed according to the traffic control strategy to control the traffic distribution and routing selection in the network. For example, network resources such as bandwidth and server load are intelligently allocated according to the real-time network traffic situation, so as to avoid waste and idleness of resources and improve the overall resource utilization. Through dynamic scheduling, it can ensure that the load of each server or network node is relatively balanced, avoid service interruption or delay caused by overloading of some nodes, improve the stability and reliability of the system, and automatically adjust the resource scale according to the system load and traffic situation to cope with the needs of sudden traffic or business growth, thereby improving the flexibility and scalability of the system.

[0056] In summary, the present invention preprocesses the collected network status information, uses the characteristic function to extract features of the preprocessed network status information, and obtains a feature vector set; based on the feature vector set, an ant colony algorithm is used for iterative calculation to obtain an optimal routing path; the optimal routing path is input into a reinforcement learning model, a flow control strategy is output, and network flow control is performed according to the flow control strategy; the present invention combines the ant colony algorithm with reinforcement learning, uses the ant colony algorithm to find the optimal flow routing, and uses reinforcement learning to dynamically adjust the flow control strategy according to changes in the network state, and realizes effective management and control of network flow in an intelligent way, which can improve the accuracy and stability of network flow prediction, realize balanced distribution and efficient utilization of network flow, and thus improve the overall performance of the network.

[0057] In an optional implementation, the feature extraction of the preprocessed network status information using a feature function to obtain a feature vector set includes:

[0058] The pre-processed network status information is subjected to feature extraction using a data packet size feature value extraction function, a data packet speed feature value extraction function, and a data packet direction feature value extraction function, respectively, to obtain a feature vector set, the feature vector set including a data packet size feature value, a data packet speed feature value, and a data packet direction feature value, and the mathematical formula is as follows:

[0059]

[0060] Among them, S(t) represents the characteristic value of the packet size, V(t) represents the characteristic value of the packet speed, D(t) represents the characteristic value of the packet direction, and L i Indicates the length of the i-th data packet, R i represents the transmission rate of the ith data packet, O i represents the direction of the ith data packet, and n represents the number of data packets.

[0061] Specifically, the preprocessed network status information is subjected to feature extraction using a feature function. Exemplarily, the feature function includes a data packet size feature value extraction function, a data packet speed feature value extraction function, and a data packet direction feature value extraction function. Feature extraction is performed respectively to obtain a data packet size feature value, a data packet speed feature value, and a data packet direction feature value. The mathematical formula is as follows:

[0062]

[0063] Among them, S(t) represents the characteristic value of the packet size, V(t) represents the characteristic value of the packet speed, D(t) represents the characteristic value of the packet direction, and L i Indicates the length of the i-th data packet, R i represents the transmission rate of the ith data packet, O i represents the direction of the ith data packet, and n represents the number of data packets.

[0064] It can be understood that the packet size characteristic value refers to the sum of all packet lengths within a certain time period, and this characteristic value can reflect the overall size of network traffic. The packet speed characteristic value refers to the sum of all packet transmission rates within the same time period, and this characteristic value can reflect the transmission rate of network traffic and possible network congestion. The packet direction characteristic value refers to the sum of all packet directions within the same time period, and this characteristic value can reflect the direction of network traffic and possible network attacks or abnormal behaviors. By extracting these three characteristic values, we can have a more comprehensive understanding of network status information, and then monitor and analyze the network more accurately.

[0065] In an optional implementation, the iterative calculation based on the feature vector set using an ant colony algorithm to obtain the best routing path includes:

[0066] Step 1: Initialize the algorithm parameters of the ant colony algorithm; wherein the algorithm parameters include the number of ants, the first maximum number of iterations, the initial value of the pheromone concentration, and the heuristic factor;

[0067] Step 2: Based on the feature vector set, randomly generate an initial path for each ant;

[0068] Step 3: According to the pheromone concentration and heuristic factor of the selected path, the next path is selected for the ant based on the path selection probability formula;

[0069] Step 4: Determine whether the current number of iterations reaches the first maximum number of iterations. If not, repeat the above step 3 until the current number of iterations reaches the first maximum number of iterations. If so, output the best routing path of the ant colony.

[0070] Specifically, based on the feature vector set, the ant colony algorithm is used for iterative calculation to obtain the best routing path. The ant colony algorithm (Ant Colony Optimization, ACO) is a heuristic search algorithm that simulates the foraging behavior of ants in nature. The core idea is to find an optimal data transmission method by simulating the behavior pattern of ants. In the ant colony algorithm, each ant will randomly explore the problem space and update the information on the path according to the results of the exploration. This information update mechanism allows ants to gradually discover the best solution and improve the efficiency of finding the best solution through a symbiotic way.

[0071] In the process of using the ant colony algorithm to iteratively calculate the best routing path, the basic parameters of the ant colony algorithm are first set, including the number of ants, the first maximum number of iterations, the initial value of the pheromone concentration, and the heuristic factor. The number of ants determines the number of ant individuals participating in the search. The more ants there are, the wider the search range may be. The first maximum number of iterations is one of the conditions for the algorithm to stop searching. At the beginning of the algorithm, the pheromone concentration on each path is set to the initial value of the pheromone concentration. The pheromone concentration reflects the quality of the path. The higher the concentration, the more likely the path is to be selected. The heuristic factor is used to consider the heuristic information of the path when calculating the path selection probability.

[0072] Furthermore, based on the feature vector set, an initial path is randomly generated for each ant. The ant selects the next path to take according to the pheromone concentration and heuristic information on the path. The path selection follows a probabilistic selection mechanism. The higher the pheromone concentration, the greater the probability of being selected. The path selection mathematical model of the ant colony algorithm can be expressed by the following formula:

[0073] P ij (t+1)=P ij (t)×(1-α)+τij ×ΔP ij ;

[0074] Among them, P ij (t+1) represents the probability of the ant moving from node i to node j at time t+1, P ij (t) represents the probability of an ant moving from node i to node j at time t, α is the decay factor, τ ij is the pheromone concentration, ΔP ij is the difference in probability updates.

[0075] Repeat the above steps until the current number of iterations reaches the first maximum number of iterations or meets other preset termination conditions. During the selection process, pheromones will continue to evaporate and enhance to reflect changes in network status. When the current number of iterations reaches the first maximum number of iterations, the optimal routing path of the ant colony is output.

[0076] In an optional implementation, the step of inputting the optimal routing path into a reinforcement learning model and outputting a flow control strategy includes:

[0077] Step 1: input the optimal routing path as an initial strategy into the reinforcement learning model, and initialize the second maximum number of iterations;

[0078] Step 2: Record the current network state, select the next action according to the initial strategy, and execute the action, and calculate the reward obtained after executing the action according to the preset reward function;

[0079] Step 3: Using a reinforcement learning algorithm to update the initial strategy;

[0080] Step 4: Determine whether the current number of iterations has reached the second maximum number of iterations. If not, repeat steps 2 and 3 until the current number of iterations reaches the second maximum number of iterations. If so, perform a performance evaluation on the updated strategy output in step 3 to obtain a first performance evaluation result, and output a final flow control strategy based on the first performance evaluation result.

[0081] Specifically, the optimal routing path is input into the reinforcement learning model, and the reinforcement learning model outputs the traffic control strategy. The reinforcement learning model gradually optimizes the traffic control strategy through continuous interaction with the environment and trial-and-error learning, so that it can adaptively respond to changes in network status, thereby achieving efficient management and control of network traffic. In this process, the state of the environment, the actions of the agent, and the design of the reward function are key factors, which directly determine the direction and effect of the agent's learning.

[0082] First, the environment of the reinforcement learning model is initialized, and the state space, action space and reward function are defined. The state space includes the state information of the network, such as node load, link bandwidth utilization and traffic distribution, etc. The action space is a set of actions that the agent can take, such as the selection of traffic routing and the traffic distribution ratio, etc. The reward function is designed to quantify the degree of improvement of network performance. For example, it can be defined according to indicators such as network throughput, delay and packet loss rate. For example, the mathematical model of the reward function used in the embodiment of the present invention is:

[0083] Q π (s,a)=E[R t |S t =s,A t =a];

[0084] Among them, S t is the observation state, A t For the action to be performed, R t For the reward, Q π (s,a) indicates that the agent starts from state s, performs action a according to strategy π, and finally obtains the reward, and t is the time step.

[0085] The best routing path output by the ant colony algorithm is used as the initial strategy for reinforcement learning, where the strategy is only related to the current state and has nothing to do with the state before the current state. The agent observes the current network state and selects an action according to the current strategy, such as adjusting the traffic routing or distribution ratio, executes the action in the network, observes the changes in the network state, and calculates the reward obtained after executing the action according to the reward function.

[0086] Furthermore, the current state, action, reward, and new state are stored as an experience, usually called a transition or episode. A batch of samples are randomly selected from the stored experience to update the value function or strategy of the agent. According to the reinforcement learning algorithm used, such as Q-learning, SARSA, or Policy Gradients, the value function or strategy parameters are updated. The update function used is as follows:

[0087] Q(s,a)←Q(s,a)+α[r+γmax a′ Q(s′,a′)-Q(s,a)];

[0088] Among them, α is the learning rate and γ is the reward decay coefficient.

[0089] Repeat steps 2 and 3 to allow the intelligent agent to learn and optimize the flow control strategy in continuous interaction with the network environment. During the iteration process, the learning rate can be gradually reduced to make the strategy update more stable. After each iteration, determine whether the current number of iterations has reached the second maximum number of iterations. If not, repeat steps 2 and 3 until the current number of iterations reaches the second maximum number of iterations. If so, perform a performance evaluation on the updated strategy output in step 3, such as network throughput, latency and other indicators, to measure the performance of the strategy under a given network state, obtain a first performance evaluation result, and output the final flow control strategy based on the first performance evaluation result.

[0090] In an optional implementation manner, outputting a final flow control strategy according to the first performance evaluation result includes:

[0091] If the first evaluation result does not meet the preset requirements, repeat steps 2 and 3 until the first evaluation result meets the preset requirements;

[0092] If the first performance evaluation result meets the preset requirements, the updated strategy is determined as the final flow control strategy and output.

[0093] Specifically, the first performance evaluation result is compared with the preset requirements. The preset requirements may be determined based on business needs, network performance standards or historical data. If the first performance evaluation result does not meet the preset requirements, it means that the performance of the current strategy still has room for improvement. Repeat steps 2 and 3, that is, continue to record network status, select actions, execute actions, calculate rewards, and use reinforcement learning algorithms to update strategies until the first performance evaluation result meets the preset requirements.

[0094] If the first performance evaluation result meets the preset requirements, it means that the performance of the current strategy has reached the standard, and the updated strategy is determined as the final flow control strategy and output.

[0095] In an optional implementation, after performing network traffic control according to the traffic control strategy, the method further includes:

[0096] The network status is monitored in real time, and real-time network status information is obtained, and the flow control strategy is adjusted based on the real-time network status information.

[0097] Specifically, after the implementation of the traffic control strategy, the network status is monitored in real time, and the real-time network status information obtained is compared with the preset performance threshold or standard. If it is found that the network status deviates from the preset performance range, or abnormal network behavior occurs, the current traffic control strategy needs to be adjusted to ensure that the network performance can remain stable and optimized in the face of continuous changes.

[0098] In an optional embodiment, the method further includes:

[0099] After adjusting the flow control strategy, performing a performance evaluation on the adjusted flow control strategy to obtain a second performance evaluation result;

[0100] When the second performance evaluation result does not meet the preset requirement, feeding back the adjusted flow control strategy to the reinforcement learning model, so that the reinforcement learning model performs model adjustment according to the second performance evaluation result;

[0101] When the second performance evaluation result meets the preset requirements, network traffic control is performed according to the adjusted traffic control strategy, and the latest network status information and the adjusted traffic control strategy are visualized and returned to relevant technical personnel.

[0102] Specifically, after adjusting the traffic control strategy, a performance evaluation is performed on the adjusted traffic control strategy to obtain a second performance evaluation result, and the second performance evaluation result is compared with the preset requirements. If the evaluation result does not meet the preset requirements, it means that the performance of the new strategy still has room for improvement, and the adjusted traffic control strategy needs to be fed back to the reinforcement learning model so that the model can be further adjusted and optimized according to the evaluation results. If the evaluation result meets the preset requirements, it means that the performance of the new strategy has met the standards, and network traffic control is performed according to the adjusted traffic control strategy. At the same time, the latest network status information and the adjusted traffic control strategy are visualized to provide clear and accurate information to technical personnel so that relevant technical personnel can intuitively understand the network status and policy effects and manage them.

[0103] It is understandable that the embodiment of the present invention also provides a user interface that is convenient for system administrators and operators to use, and relevant technical personnel can use the user interface to configure system parameters, view system status, monitor network traffic, and other tasks.

[0104] In a second aspect, an embodiment of the present invention provides a dynamic network traffic control device, see Figure 2 , which is a structural diagram of an embodiment of a dynamic network traffic control device provided by the present invention.

[0105] like Figure 2 As shown, the device comprises:

[0106] The network status feature extraction module 21 is used to pre-process the collected network status information, and extract features from the pre-processed network status information using a feature function to obtain a feature vector set;

[0107] The best routing path generation module 22 is used to obtain the best routing path by iterative calculation using an ant colony algorithm based on the feature vector set;

[0108] The flow control strategy generation module 23 is used to input the optimal routing path into the reinforcement learning model, output the flow control strategy, and perform network flow control according to the flow control strategy.

[0109] In an optional implementation, the network status feature extraction module 21 is further used for:

[0110] The data packet size feature value extraction function, data packet speed feature value extraction function and data packet direction feature value extraction function are respectively used to extract features of the pre-processed network status information to obtain the data packet size feature value, data packet speed feature value and data packet direction feature value. The mathematical formula is as follows:

[0111]

[0112] Among them, S(t) represents the characteristic value of the packet size, V(t) represents the characteristic value of the packet speed, D(t) represents the characteristic value of the packet direction, and L i Indicates the length of the i-th data packet, R i represents the transmission rate of the ith data packet, O i represents the direction of the ith data packet, and n represents the number of data packets.

[0113] In an optional implementation, the optimal routing path generation module 22 is further used to:

[0114] Step 1: Initialize the algorithm parameters of the ant colony algorithm; wherein the algorithm parameters include the number of ants, the first maximum number of iterations, the initial value of the pheromone concentration, and the heuristic factor;

[0115] Step 2: Based on the feature vector set, randomly generate an initial path for each ant;

[0116] Step 3: According to the pheromone concentration and heuristic factor of the selected path, the next path is selected for the ant based on the path selection probability formula;

[0117] Step 4: Determine whether the current number of iterations reaches the first maximum number of iterations. If not, repeat the above step 3 until the current number of iterations reaches the first maximum number of iterations. If so, output the best routing path of the ant colony.

[0118] In an optional implementation manner, the flow control strategy generating module 23 is further used to:

[0119] Step 1: input the optimal routing path as an initial strategy into the reinforcement learning model, and initialize the second maximum number of iterations;

[0120] Step 2: Record the current network state, select the next action according to the initial strategy, and execute the action, and calculate the reward obtained after executing the action according to the preset reward function;

[0121] Step 3: Using a reinforcement learning algorithm to update the initial strategy;

[0122] Step 4: Determine whether the current number of iterations has reached the second maximum number of iterations. If not, repeat steps 2 and 3 until the current number of iterations reaches the second maximum number of iterations. If so, perform a performance evaluation on the updated strategy output in step 3 to obtain a first performance evaluation result, and output a final flow control strategy based on the first performance evaluation result.

[0123] In an optional implementation manner, the flow control strategy generating module 23 is further used to:

[0124] If the first evaluation result does not meet the preset requirements, repeat steps 2 and 3 until the first evaluation result meets the preset requirements;

[0125] If the first performance evaluation result meets the preset requirements, the updated strategy is determined as the final flow control strategy and output.

[0126] In an optional implementation, the device further includes a flow control strategy adjustment module, which is used to:

[0127] The network status is monitored in real time, and real-time network status information is obtained, and the flow control strategy is adjusted based on the real-time network status information.

[0128] In an optional implementation, the device further includes a strategy performance evaluation module, which is used to:

[0129] After adjusting the flow control strategy, performing a performance evaluation on the adjusted flow control strategy to obtain a second performance evaluation result;

[0130] When the second performance evaluation result does not meet the preset requirement, feeding back the adjusted flow control strategy to the reinforcement learning model, so that the reinforcement learning model performs model adjustment according to the second performance evaluation result;

[0131] When the second performance evaluation result meets the preset requirements, network traffic control is performed according to the adjusted traffic control strategy, and the latest network status information and the adjusted traffic control strategy are visualized and returned to relevant technical personnel.

[0132] In a third aspect, an embodiment of the present invention provides an electronic device, see Figure 3 , which is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0133] like Figure 3 As shown, the device includes:

[0134] A memory 31, used for storing computer programs;

[0135] A processor 32, configured to execute the computer program;

[0136] Wherein, when the processor 32 executes the computer program, the dynamic network traffic control method as described in any of the above embodiments is implemented.

[0137] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 32 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0138] The processor 32 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0139] The memory 31 can be used to store the computer program and / or module, and the processor 32 realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory 31, and calling the data stored in the memory 31. The memory 31 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 31 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0140] It should be noted that the above electronic device includes, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 3 The structural diagram is merely an example of the electronic device described above and does not constitute a limitation on the electronic device, and may include more components than shown in the figure, or a combination of certain components, or different components.

[0141] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the dynamic network traffic control method described in any of the above embodiments is implemented.

[0142] It should be understood that the present invention can implement all or part of the processes in the above-mentioned dynamic network traffic control method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned dynamic network traffic control method. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0143] The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. It should be pointed out that for those skilled in the art, several equivalent obvious variations and / or equivalent substitutions can be made without departing from the technical principles of the present invention. These obvious variations and / or equivalent substitutions should also be regarded as the protection scope of the present invention.

Claims

1. A dynamic network traffic control method, characterized in that: include: Preprocessing the collected network status information, extracting features from the preprocessed network status information using a feature function, and obtaining a feature vector set; Based on the feature vector set, an ant colony algorithm is used for iterative calculation to obtain an optimal routing path; The optimal routing path is input into a reinforcement learning model, a flow control strategy is output, and network flow control is performed according to the flow control strategy.

2. The dynamic network traffic control method according to claim 1, characterized in that: The feature function is used to extract features from the preprocessed network status information to obtain a feature vector set, including: The data packet size feature value extraction function, data packet speed feature value extraction function and data packet direction feature value extraction function are respectively used to extract features of the pre-processed network status information to obtain the data packet size feature value, data packet speed feature value and data packet direction feature value. The mathematical formula is as follows: Among them, S(t) represents the characteristic value of the packet size, V(t) represents the characteristic value of the packet speed, D(t) represents the characteristic value of the packet direction, and L i Indicates the length of the i-th data packet, R i represents the transmission rate of the ith data packet, O i represents the direction of the ith data packet, and n represents the number of data packets.

3. The dynamic network traffic control method according to claim 1, characterized in that: The method of iteratively calculating based on the feature vector set using an ant colony algorithm to obtain an optimal routing path includes: Step 1: Initialize the algorithm parameters of the ant colony algorithm; wherein the algorithm parameters include the number of ants, the first maximum number of iterations, the initial value of the pheromone concentration, and the heuristic factor; Step 2: Based on the feature vector set, randomly generate an initial path for each ant; Step 3: According to the pheromone concentration and heuristic factor of the selected path, the next path is selected for the ant based on the path selection probability formula; Step 4: Determine whether the current number of iterations reaches the first maximum number of iterations. If not, repeat the above step 3 until the current number of iterations reaches the first maximum number of iterations. If so, output the best routing path of the ant colony.

4. The dynamic network traffic control method according to claim 1, characterized in that: The step of inputting the optimal routing path into a reinforcement learning model and outputting a flow control strategy includes: Step 1: input the optimal routing path as an initial strategy into the reinforcement learning model, and initialize the second maximum number of iterations; Step 2: Record the current network state, select the next action according to the initial strategy, and execute the action, and calculate the reward obtained after executing the action according to the preset reward function; Step 3: Using a reinforcement learning algorithm to update the initial strategy; Step 4: Determine whether the current number of iterations has reached the second maximum number of iterations. If not, repeat steps 2 and 3 until the current number of iterations reaches the second maximum number of iterations. If so, perform a performance evaluation on the updated strategy output in step 3 to obtain a first performance evaluation result, and output a final flow control strategy based on the first performance evaluation result.

5. The dynamic network traffic control method according to claim 4, characterized in that: Outputting a final flow control strategy according to the first performance evaluation result includes: If the first evaluation result does not meet the preset requirements, repeat steps 2 and 3 until the first evaluation result meets the preset requirements; If the first performance evaluation result meets the preset requirements, the updated strategy is determined as the final flow control strategy and output.

6. The dynamic network traffic control method according to claim 1, characterized in that: After the network traffic control is performed according to the traffic control strategy, the method further includes: The network status is monitored in real time, and real-time network status information is obtained, and the flow control strategy is adjusted based on the real-time network status information.

7. The dynamic network traffic control method according to claim 6, characterized in that: The method further comprises: After adjusting the flow control strategy, performing a performance evaluation on the adjusted flow control strategy to obtain a second performance evaluation result; When the second performance evaluation result does not meet the preset requirement, feeding back the adjusted flow control strategy to the reinforcement learning model, so that the reinforcement learning model performs model adjustment according to the second performance evaluation result; When the second performance evaluation result meets the preset requirements, network traffic control is performed according to the adjusted traffic control strategy, and the latest network status information and the adjusted traffic control strategy are visualized and returned to relevant technical personnel.

8. A dynamic network traffic control device, characterized in that: include: The network status feature extraction module is used to pre-process the collected network status information, and use the feature function to extract the features of the pre-processed network status information to obtain a feature vector set; An optimal routing path generation module is used to obtain an optimal routing path by iterative calculation using an ant colony algorithm based on the feature vector set; The flow control strategy generation module is used to input the optimal routing path into the reinforcement learning model, output the flow control strategy, and perform network flow control according to the flow control strategy.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program; Wherein, when the processor executes the computer program, the dynamic network traffic control method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the dynamic network traffic control method according to any one of claims 1 to 7 is implemented.

Images