A low-delay and high-efficiency communication method and device for a switch

Abstract

The present invention discloses a low-latency and efficient communication method and device for a switch. The method adopts a deep reinforcement learning network structure for intelligent scheduling, including obtaining the switch operation status information, inputting the status information into an intelligent scheduling model composed of a state encoder, a policy network and a value network to generate a scheduling strategy, executing the scheduling strategy and optimizing the model parameters by reward evaluation based on task completion time, system energy consumption, resource utilization and network delay. The device includes a processor and a memory to realize the functions of state acquisition, scheduling decision and model optimization, and is provided with an FPGA acceleration circuit, including a multi-stage pipeline parsing circuit and a hardware acceleration circuit. The present invention solves the technical problem that the traditional switch scheduling method is difficult to balance system performance, resource utilization and energy consumption, realizes the adaptive optimization of the scheduling strategy through deep reinforcement learning and hardware acceleration, significantly improves the real-time and efficiency of communication, and reduces the system energy consumption.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a low-latency and efficient communication method and system for a switch based on deep reinforcement learning. Background Art

[0002] With the rapid development of 5G, IoT, cloud computing and artificial intelligence technologies, network traffic is experiencing explosive growth, diversified business types and differentiated service quality requirements. As the core equipment in the network, the communication efficiency and service quality of the switch directly affect the performance of the entire network system. At present, high-performance switches usually use multi-stage pipeline architecture, multi-core parallel processing technology and programmable data plane technologies to improve data processing capabilities, but they still face many challenges in the face of increasingly complex network environments.

[0003] The existing technology mainly has the following problems:

[0004] 1. Although the scheduling strategy of the switch supports multiple modes, it mainly relies on preset scheduling algorithms, such as polling and priority scheduling. This scheduling mechanism based on fixed rules is difficult to achieve optimal task allocation in complex scenarios such as sudden network traffic and drastic fluctuations in business load. Especially in the case of concurrent processing of multiple businesses, the existing scheduling strategy is difficult to dynamically adjust the processing order and resource allocation ratio according to the real-time network status, which easily leads to performance bottlenecks.

[0005] 2. Although the existing resource management mechanism supports dynamic allocation, the optimization strategy is relatively simple and it is difficult to cope with complex load scenarios. Especially when diversified services are processed concurrently, the existing resource scheduling mechanism is difficult to achieve accurate load prediction and intelligent resource allocation. This results in the overall system performance not being fully utilized when some processing units are overloaded while other processing units are relatively idle.

[0006] 3. Although existing switches have multi-dimensional performance monitoring capabilities, the optimization of each indicator is often carried out independently, lacking a unified end-to-end service quality assurance mechanism. In the actual operation process, there are complex interactions between multiple performance indicators such as task completion time, system energy consumption, resource utilization, and network latency. The existing single-dimensional optimization strategy is difficult to achieve coordinated optimization of multiple indicators and cannot meet the comprehensive requirements of modern network applications for service quality.

[0007] 4. Although existing switches have basic status monitoring capabilities, they lack intelligent analysis and prediction decision-making mechanisms based on historical data. When facing a dynamically changing network environment, making decisions based solely on real-time monitoring data often lags behind changes in network status. Due to the lack of deep mining and trend analysis capabilities for historical data, it is difficult to achieve trend prediction and active optimization of network performance, which affects the system's ability to respond quickly to network changes.

[0008] 5. Although existing switches use programmable hardware architecture, the hardware support for intelligent scheduling and dynamic optimization is still insufficient. In particular, there is a lack of dedicated hardware acceleration design for artificial intelligence algorithms such as deep learning, which makes it impossible to efficiently support complex optimization decisions at the hardware level. This separation of software and hardware functions limits the processing efficiency of switches in complex business scenarios.

[0009] Therefore, it is urgent to propose a switch communication method that can achieve intelligent scheduling and dynamic optimization, and establish an end-to-end performance optimization mechanism through artificial intelligence technologies such as deep reinforcement learning and combined with software and hardware collaborative design. This method not only needs to realize intelligent scheduling decisions at the algorithm level, but also needs to provide corresponding acceleration support at the hardware level, so as to meet the urgent needs of modern network applications for efficient and low-latency communication. Summary of the invention

[0010] The problem to be solved by the present invention is how to provide a low-latency and efficient switch communication method and device to achieve intelligent task scheduling and resource allocation.

[0011] In order to solve the above technical problems, the technical solution adopted by the present invention is: a low-latency and high-efficiency communication method of a switch, characterized by comprising the following steps:

[0012] S1 obtains switch operation status information, the operation status information includes queue status information, resource status information and performance status information;

[0013] S2. Input the operation status information into the intelligent scheduling model for analysis and generating a scheduling strategy, wherein the intelligent scheduling model adopts a deep reinforcement learning network structure, including a state encoder for extracting time series features, a strategy network for generating actions, and a value network for evaluating action values;

[0014] S3. Execute the scheduling strategy and perform reward evaluation based on task completion time, system energy consumption, resource utilization and network delay, and optimize the network parameters of the intelligent scheduling model according to the reward evaluation results.

[0015] A further technical solution is that the operating status information acquired by S1 specifically includes:

[0016] Queue status information, including the task queue length and waiting time of each computing device;

[0017] Resource status information, including processor utilization and memory usage of each computing device;

[0018] Performance metrics information, including energy consumption levels and current latency for each computing device.

[0019] A further technical solution is that the S2 intelligent scheduling model adopts a deep reinforcement learning network structure, including a state encoder for extracting temporal features, a policy network for generating actions, and a value network for evaluating the value of actions;

[0020] Among them, the state encoder adopts a long short-term memory network LSTM structure; the policy network and value network adopt an Actor-Critic architecture, the policy network is used as the Actor to generate the probability distribution of scheduling actions, and the value network is used as the Critic to evaluate the value function of the state-action pair.

[0021] A further technical solution is that the LSTM structure of the state encoder includes:

[0022] The input layer is used to construct the queue status information, multi-level cache status information, resource status information and performance indicator information into an input vector x at time t t ;

[0023] The gated unit layer includes a forget gate for t and the previous hidden state h t-1 Selectively forget historical information; input gate, used to control the update weight of current state information; output gate, used to adjust the output of state features;

[0024] The state update unit updates the cell state c according to the output of the gating unit t , maintain long-term performance trend information; update the hidden state h t , reflecting the current system status;

[0025] The output layer is used to t Generate a state feature vector s containing processor occupancy, memory usage, task queue status, network transmission delay, and bandwidth occupancy t .

[0026] A further technical solution is that the implementation process of the Actor-Critic architecture specifically includes:

[0027] Construct a policy network to generate an action selection strategy, whose input layer receives state information s t , after nonlinear transformation, the output action selection probability distribution

[0028] Construct a value network to evaluate the state-action value, whose input layer receives the state s t and action a t Information, output state-action value function after nonlinear transformation

[0029] An exploration-utilization balanced action selection strategy is adopted to execute the actions output by the policy network. The initial exploration probability and the minimum exploration probability are set, and the exploration probability is gradually reduced during the training process. In this way, a high exploration probability is maintained in the early stage of training to expand the policy search space, and the utilization probability is gradually increased in the later stage of training to improve the certainty of the policy.

[0030] A further technical solution is that the optimization process of S3 includes:

[0031] Calculate the task completion time d based on the task queue length and waiting time t , accumulate the real-time power consumption of each computing device to get the total energy consumption of the system e t , calculate resource utilization balance based on the standard deviation of processor and memory utilization j t ; The performance index q is calculated based on the ratio of the current actual delay to the target delay t ;

[0032] Based on task completion time d t 、Total system energy consumption t , resource utilization balance t and performance index q t Calculate the immediate reward rt, the formula is:

[0033]

[0034] Among them, d t0 is the preset target task completion time, e t0 is the preset target system energy consumption;

[0035] The policy gradient method is used to optimize the policy network parameters and the policy is updated by maximizing the objective function;

[0036] Temporal difference learning is used to optimize the value network parameters and update the value evaluation by minimizing the loss function.

[0037] The present invention also discloses a low-delay and high-efficiency communication device for a switch, comprising:

[0038] A processor for running an intelligent scheduler;

[0039] FPGA acceleration device, used to accelerate data plane processing;

[0040] A memory, connected to the processor, for storing the intelligent scheduling program;

[0041] The intelligent scheduling program is implemented when executed by the processor:

[0042] Status collection function, used to periodically obtain and cache switch operation status information;

[0043] A scheduling decision function, which analyzes the operation status information based on a deep reinforcement learning network model and generates a scheduling strategy;

[0044] The model optimization function optimizes the deep reinforcement learning network model based on the reward value of the execution feedback.

[0045] A further technical solution is that the device realizes data plane processing acceleration through FPGA, including:

[0046] Multi-stage pipeline parsing circuit, used for parallel processing of parsing of input data packets, to achieve fast parsing and classification of messages;

[0047] A reconfigurable parallel processing circuit, connected to the pipeline parsing circuit, having multiple parallel processing channels and a load balancing circuit between channels, for parallel forwarding of parsed data packets;

[0048] A data plane acceleration circuit, connected to the reconfigurable parallel processing circuit, including a forwarding table lookup circuit and a hardware queue scheduling circuit, for accelerating forwarding decisions and queue scheduling of data packets;

[0049] The real-time monitoring circuit is used to collect performance indicators and resource status such as data packet processing delay, queue length, channel utilization, etc., and output the monitoring data to the intelligent scheduling model through the status feedback circuit.

[0050] The beneficial effects of the present invention are as follows:

[0051] (1) The present invention uses the LSTM network structure to extract time series features, which can effectively memorize and analyze historical load change trends, avoid the system's excessive response to short-term load fluctuations, and improve the stability of scheduling decisions. The dual-network architecture based on Actor-Critic separates strategy generation and evaluation, significantly reduces the variance of strategy search through parallel optimization, accelerates the convergence of the model, and enables the system to learn the optimal scheduling strategy more quickly.

[0052] (2) The FPGA hardware acceleration architecture of the present invention is used as a data plane processing acceleration solution. It realizes parallel parsing and processing of data packets through multi-stage pipelines, significantly reducing processing delays. Reconfigurable parallel processing channels and load balancing circuits ensure the dynamic allocation and efficient utilization of system resources, and improve the throughput of data processing. The hardware-level forwarding table lookup circuit and queue scheduling circuit directly complete data packet forwarding and queue scheduling at the hardware level, reducing processing overhead and further optimizing system delays. At the same time, the performance indicators and resource status collected by the real-time monitoring circuit can provide decision-making basis for the intelligent scheduling model.

[0053] (3) The present invention realizes efficient resource scheduling through software and hardware collaboration. Real-time monitoring of queue status information effectively reduces the average waiting time of tasks. The combination of hardware acceleration circuit and intelligent scheduling algorithm greatly improves data processing efficiency and optimizes system energy consumption. The overall architecture has good scalability, which makes it easy to adjust processing capacity according to actual needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the implementation of the present invention or the existing technical solutions, the following briefly introduces the drawings required to be used in the embodiments or the description of the prior art.

[0055] Figure 1 The present invention is a flow chart of a low-latency and efficient communication method for switches.

[0056] Figure 2 This is the structure diagram of the intelligent scheduling model.

[0057] Figure 3 This is a diagram of the device architecture. DETAILED DESCRIPTION

[0058] The above detailed description is a specific description of a feasible embodiment of the present invention. The embodiment is not intended to limit the patent scope of the present invention. Any equivalent implementation or modification that does not deviate from the present invention should be included in the patent scope of this case.

[0059] Reference Figure 1 The main steps of the low-latency and efficient communication method for switches provided by the present invention are as follows:

[0060] (1) obtaining switch operation status information, wherein the operation status information includes queue status information, resource status information, and performance status information;

[0061] (2) inputting the operation status information into an intelligent scheduling model for analysis and generating a scheduling strategy;

[0062] (3) executing the scheduling strategy and performing reward evaluation based on task completion time, system energy consumption, resource utilization and network delay, and optimizing the network parameters of the intelligent scheduling model according to the reward evaluation results.

[0063] As a preferred embodiment of the present invention, in step (1), the process of obtaining the switch operation status information is as follows:

[0064] (1-1) First, the queue status information is obtained. The system deploys a monitoring program on each computing device to periodically collect the length of the task queue and record the number of tasks currently waiting to be processed. At the same time, the monitoring program tracks the waiting time of each task and calculates the longest waiting time and average waiting time in the task queue. These queue status data will be stored in the status cache for subsequent analysis of the dynamic change characteristics of the queue.

[0065] (1-2) After obtaining the queue status, the system continues to collect resource status information. The resource monitoring module collects the processor utilization data of each computing device in real time, including the real-time CPU usage and the load distribution of each core. At the same time, the monitoring module also records the memory usage status, and calculates the accurate memory usage by detecting the used memory size and the remaining available memory capacity. This resource status information will be used to evaluate the load balancing of the system.

[0066] (1-3) Finally, the system will collect performance indicator information. The performance monitoring module uses built-in energy consumption sensors to measure the energy consumption level of each computing device during task processing in real time. At the same time, the network detection module measures the current network transmission delay. These performance data can intuitively reflect the operating efficiency of the system and provide an important basis for subsequent scheduling optimization.

[0067] (1-4) As a preferred embodiment, in order to ensure the real-time and accuracy of the status information, the system will update the status data regularly. By setting a reasonable sampling period, the status cache is updated in time when new status information arrives. The system will also pre-process the collected raw data, including data cleaning, normalization and other operations, to ensure that different types of status information can be effectively processed by the intelligent scheduling model.

[0068] (1-5) After completing the collection and processing of the above status information, the system integrates the queue status information, resource status information and performance indicator information to construct a complete state vector. This state vector will be used as the input of the intelligent scheduling model for subsequent scheduling strategy generation.

[0069] As a preferred embodiment of the present invention, refer to Figure 2In step (2), the process of inputting the operation status information into the intelligent scheduling model for analysis and generating the scheduling strategy is as follows:

[0070] (2-1) The present invention adopts a deep reinforcement learning network as the core architecture of the intelligent scheduling model. The model consists of three key components: a state encoder for extracting time series features, a policy network for generating actions, and a value network for evaluating the value of actions. As a preferred embodiment of the present invention, the state encoder adopts a two-layer LSTM structure, each layer contains 128 hidden units; the policy network adopts a three-layer fully connected neural network structure, and the number of neurons in each layer is 256, 128 and 64 respectively; the value network also adopts a three-layer fully connected structure, and the number of neurons in each layer is 256, 128 and 1. This network configuration has shown good feature extraction capabilities and decision-making performance in experiments.

[0071] (2-2) In the specific implementation of the state encoder, the system state information is first encoded into an input vector of fixed dimension. As a preferred embodiment of the present invention, the input vector xt contains 64-dimensional features, of which the queue state information occupies 16 dimensions, the resource state information occupies 32 dimensions, and the performance indicator information occupies 16 dimensions. The gating unit of LSTM adopts a sigmoid activation function with a value range of [0,1] to control the flow of information. As a preferred embodiment, the bias value of the forget gate is initialized to 1.0 to retain more historical information; the bias values ​​of the input gate and the output gate are initialized to 0.0, so that the model pays more attention to the learning of new information in the early stage of training. The state update unit adopts the tanh activation function so that the updated state value is distributed in the interval [-1,1], which is conducive to the stable propagation of the gradient.

[0072] (2-3) In the Actor-Critic architecture, the design of the policy network and the value network is also based on a large number of experimental optimizations. As a preferred embodiment of the present invention, the three fully connected layers of the policy network use ReLU, ReLU and Softmax activation functions respectively, among which ReLU can alleviate the gradient vanishing problem, and Softmax is used to generate a normalized action probability distribution. The first two layers of the value network use the ReLU activation function, and the last layer uses a linear activation function to predict continuous state-action values. The network training uses the Adam optimizer, the initial value of the learning rate is set to 0.001, and a learning rate decay strategy is used. The learning rate decays to 0.95 times the original value every 10,000 training steps.

[0073] (2-4) In the specific implementation of the exploration strategy, the present invention adopts the ε-greedy strategy for action selection. As a preferred embodiment, the initial exploration probability ε is set to 0.9, the minimum exploration probability threshold is set to 0.1, and the decay rate is 0.995. Specifically, for every 1000 steps of training, the current exploration probability is multiplied by the decay rate until the minimum threshold is reached. At the same time, the temperature parameter τ is introduced to adjust the smoothness of the action probability distribution. The initial value is set to 1.0 and gradually decreases to 0.1 as the training progresses, so that the action selection gradually transitions from full exploration in the early stage to stable utilization in the later stage.

[0074] (2-5) In order to improve the training efficiency and generalization ability of the model, the present invention also adopts a number of optimization techniques. As a preferred embodiment, an experience replay buffer is used to store training samples, and the buffer size is set to 10,000; 256 samples are randomly sampled to form a mini-batch for each training; and the target network mechanism is adopted to update the target network parameters every 1,000 steps. At the same time, a gradient clipping mechanism is introduced to limit the gradient norm to the interval [-5,5] to prevent gradient explosion. In addition, the robustness of the model is enhanced by adding Gaussian noise (mean 0, standard deviation 0.01) to the state features.

[0075] As a preferred embodiment of the present invention, in step (3), the process of executing the scheduling strategy and performing reward evaluation and model optimization is as follows:

[0076] (3-1) The system first calculates the scheduling effect based on multiple key indicators. For the task completion time d t The present invention monitors the task queues of each computing device and counts the time interval from when a task enters the queue to when it is executed. As a preferred embodiment of the present invention, the system samples the task queue status every 100ms, records the waiting time and execution time of each task, and calculates the average task completion time in the last 10 sampling cycles. At the same time, the preset target task completion time dt0 is set to 50ms, and this parameter can be adjusted according to actual business needs.

[0077] (3-2) In terms of system energy consumption, the energy consumption data of each computing device is recorded in real time through the deployed energy consumption monitoring module. As a preferred embodiment, the sampling frequency of energy consumption monitoring is set to 1000 Hz, generating 1000 power consumption sampling points per second. The system accumulates these sampling points according to the time window (preferably 1 second) to obtain the total energy consumption of the system e t The preset target system energy consumption is t0 Determined based on historical operating data, the preferred setting is 1.2 times the lowest energy consumption level observed during operation, which not only ensures the attainability of the energy-saving target but also reserves a certain performance margin.

[0078] (3-3) For resource utilization balance jt , the present invention adopts an evaluation method based on standard deviation. Specifically, the standard deviation of processor utilization and memory utilization are calculated respectively to evaluate the distribution of resource usage. As a preferred embodiment, the target interval of resource utilization is set to [30%, 70%]. When the resource utilization of all computing devices falls within this interval, the standard deviation is the smallest, and the resource utilization balance reaches the optimal value of 1.0; as the imbalance of resource utilization distribution increases, the balance index decays according to the law of exponential function, and the minimum value is set to 0.1.

[0079] (3-4) Performance index q t The calculation of comprehensively considers the ratio of actual delay to target delay. As a preferred embodiment, the system collects network transmission delay data every 10ms and calculates the average delay of the last 100 sampling points as the current actual delay. Different thresholds are set for target delay according to the service type: for high-priority services, the target delay is set to 1ms; for ordinary services, the target delay is set to 5ms; for background services, the target delay is set to 20ms. The performance indicator qt is calculated by the ratio of the actual delay to the corresponding target delay, and is mapped to the [0.1, 1.0] interval through the sigmoid function.

[0080] (3-5) After obtaining the above evaluation indicators, the system generates an immediate reward value according to the reward calculation formula, which is:

[0081]

[0082] Among them, d t0 is the preset target task completion time, e t0 is the preset target system energy consumption;

[0083] As a preferred embodiment, in order to prevent the reward value from being too large and causing unstable training, the system normalizes the calculated reward value and limits it to the interval [-1,1]. At the same time, the reward decay factor γ (preferably 0.99) is introduced to balance the immediate reward and long-term benefits.

[0084] (3-6) Based on the calculated reward value, the present invention uses the policy gradient method to optimize the policy network parameter θ. As a preferred embodiment, an advantage function estimation method with a baseline is used, where the baseline value is the average reward of the last 100 episodes. The loss function of the policy network is the product of the negative log-likelihood loss and the advantage function. The parameters are updated by the stochastic gradient ascent method, and the learning rate is set to 0.0001. In order to improve the training stability, the gradient clipping technique is used to limit the gradient norm to the interval [-1,1].

[0085] (3-7) For the optimization of the value network parameter φ, the present invention adopts the temporal difference learning method. As a preferred embodiment, the TD(λ) algorithm is used, where the λ parameter is set to 0.95 to control the weight of multi-step returns. The loss function of the value network adopts the mean square error, and the parameters are updated through the Adam optimizer, and the learning rate is set to 0.0005. At the same time, the target network mechanism is introduced, and the target network parameters are updated every 500 steps. The update weight τ is set to 0.01, which effectively improves the stability of training.

[0086] As another embodiment (4) of the present invention, referring to Figure 3 , a low-latency and efficient communication device for a switch that implements the above method is provided. The device includes a processor, a memory connected to the processor, and an FPGA device for accelerating data plane processing. The memory is used to store an intelligent scheduler. As a preferred embodiment, the processor adopts a dual-core architecture, the main frequency is not less than 2.4GHz, and has hardware multi-threading technology; the memory includes no less than 4GB of running memory and no less than 32GB of persistent storage space to ensure the efficient operation of the intelligent scheduler. When the intelligent scheduler runs on the processor, it mainly implements the following functional modules:

[0087] (4-1) Status collection function module, responsible for periodically acquiring and caching switch operation status information. As a preferred embodiment, this module adopts a multi-threaded parallel collection mechanism, and sets 4 collection threads to be responsible for different types of status information acquisition. Among them, the first thread is responsible for queue status collection, with a sampling period of 10ms; the second thread is responsible for resource status collection, with a sampling period of 100ms; the third thread is responsible for performance indicator collection, with a sampling period of 1ms; the fourth thread is responsible for data preprocessing and cache management, with a processing period of 50ms. Each thread uses a ring buffer to store the collected data, and the buffer size is dynamically adjusted according to the sampling period.

[0088] (4-2) Scheduling decision function module, which analyzes the operation status information and generates a scheduling strategy based on a deep reinforcement learning network model. As a preferred embodiment, this module adopts a pipeline architecture, including four stages: feature extraction, state encoding, action generation, and strategy output. The processing delay of each stage does not exceed 0.5ms. Through fine-grained task division and pipeline parallel processing, the real-time scheduling decision is ensured.

[0089] (4-3) Model optimization function module, responsible for optimizing the deep reinforcement learning network model based on the reward value of execution feedback. As a preferred embodiment, this module runs in an independent optimization thread and uses an experience playback buffer to store training samples with a buffer size of 100,000 records. The model optimization process adopts batch processing, with each batch consisting of 256 samples and an optimization interval of 1 second.

[0090] In order to further improve the system performance, the present invention uses FPGA to implement hardware acceleration of data plane processing. The specific implementation scheme (5) of the acceleration device is as follows:

[0091] (5-1) Multi-stage pipeline parsing circuit: As a preferred embodiment, an 8-stage pipeline structure is used to implement parallel parsing of data packets. Stages 1-2 are responsible for packet header parsing, with an operating frequency of 200MHz; stages 3-4 are responsible for field extraction, with an operating frequency of 400MHz; stages 5-6 are responsible for protocol identification, with an operating frequency of 300MHz; stages 7-8 are responsible for result encapsulation, with an operating frequency of 250MHz. Data is exchanged between each stage of the pipeline through a dual-port RAM, with a RAM depth of 1024 and a bit width of 256bit.

[0092] (5-2) Reconfigurable parallel processing circuit: As a preferred embodiment, 16 parallel processing channels are designed, each channel is equipped with an independent computing unit and local storage. The load balancing circuit between channels adopts a dynamic arbitration mechanism with an arbitration cycle of 4 clock cycles, supporting priority-based channel selection. The operating frequency of the processing channel can be dynamically adjusted within the range of 100MHz-500MHz, and changes adaptively according to the load conditions.

[0093] (5-3) Data plane acceleration circuit: As a preferred embodiment, the forwarding table search circuit adopts a three-stage pipeline structure, including index calculation, table entry matching and result output. The forwarding table capacity is 16K entries, and CAM is used to achieve fast search, and the search delay does not exceed 2 clock cycles. The hardware queue scheduling circuit adopts a priority-based polling mechanism, supports 8-level priority configuration, and the scheduling time does not exceed 4 clock cycles.

[0094] (5-4) Real-time monitoring circuit: As a preferred embodiment, a distributed monitoring architecture is adopted, and performance counters and status registers are arranged in key processing units. Monitoring data is transmitted through a dedicated state feedback bus with a bus width of 32 bits and an operating frequency of 100MHz. The state feedback circuit adopts an interrupt mode. When the monitoring index exceeds the preset threshold, an interrupt is triggered to notify the intelligent scheduler to process.

[0095] The device of the present invention implements control plane functions such as state acquisition, scheduling decision and model optimization by executing an intelligent scheduling program through a processor, and implements data plane functions such as packet parsing, parallel processing, forwarding and queue scheduling through an FPGA, thereby completing low-latency and efficient communication of the switch.

Claims

1. A low-latency and efficient communication method for a switch, characterized in that: The method comprises: S1 obtains switch operation status information, the operation status information includes queue status information, resource status information and performance status information; S2. Input the operation status information into the intelligent scheduling model for analysis and generating a scheduling strategy, wherein the intelligent scheduling model adopts a deep reinforcement learning network structure, including a state encoder for extracting time series features, a strategy network for generating actions, and a value network for evaluating action values; S3. Execute the scheduling strategy and perform reward evaluation based on task completion time, system energy consumption, resource utilization and network delay, and optimize the network parameters of the intelligent scheduling model according to the reward evaluation results.

2. A low-latency and efficient communication method for a switch according to claim 1, characterized in that: The operation status information obtained by S1 specifically includes: Queue status information, including the task queue length and waiting time of each computing device; Resource status information, including processor utilization and memory usage of each computing device; Performance metrics information, including energy consumption levels and current latency for each computing device.

3. A low-latency and efficient communication method for a switch according to claim 1, characterized in that: The S2 intelligent scheduling model adopts a deep reinforcement learning network structure, including a state encoder for extracting temporal features, a policy network for generating actions, and a value network for evaluating the value of actions; Among them, the state encoder adopts a long short-term memory network LSTM structure; the policy network and value network adopt an Actor-Critic architecture, the policy network is used as the Actor to generate the probability distribution of scheduling actions, and the value network is used as the Critic to evaluate the value function of the state-action pair.

4. A low-latency and efficient communication method for a switch according to claim 3, characterized in that: The LSTM structure of the state encoder includes: The input layer is used to construct the queue status information, multi-level cache status information, resource status information and performance indicator information into an input vector x at time t t ; The gated unit layer includes a forget gate for t and the previous hidden state h t-1 Selectively forget historical information; input gate, used to control the update weight of current state information; output gate, used to adjust the output of state features; The state update unit updates the cell state c according to the output of the gating unit t , maintain long-term performance trend information; update the hidden state h t , reflecting the current system status; The output layer is used to t Generate a state feature vector s containing processor occupancy, memory usage, task queue status, network transmission delay, and bandwidth occupancy t .

5. A low-latency and efficient communication method for a switch according to claim 3, characterized in that: The implementation process of the Actor-Critic architecture specifically includes: Construct a policy network to generate an action selection strategy, whose input layer receives state information s t , after nonlinear transformation, the output action selection probability distribution Construct a value network to evaluate the state-action value, whose input layer receives the state s t and action a t Information, output state-action value function after nonlinear transformation An exploration-utilization balanced action selection strategy is adopted to execute the actions output by the policy network. The initial exploration probability and the minimum exploration probability are set, and the exploration probability is gradually reduced during the training process. In this way, a high exploration probability is maintained in the early stage of training to expand the policy search space, and the utilization probability is gradually increased in the later stage of training to improve the certainty of the policy.

6. A low-latency and efficient communication method for a switch according to claim 1, characterized in that: The S3 optimization process includes: Calculate the task completion time d based on the task queue length and waiting time t , accumulate the real-time power consumption of each computing device to get the total energy consumption of the system e t , calculate resource utilization balance based on the standard deviation of processor and memory utilization j t ; The performance index q is calculated based on the ratio of the current actual delay to the target delay t ; Based on task completion time d t 、Total system energy consumption t , resource utilization balance t and performance index q t Calculate the immediate reward rt, the formula is: Among them, d t0 is the preset target task completion time, e t0 is the preset target system energy consumption; The policy gradient method is used to optimize the policy network parameters and the policy is updated by maximizing the objective function; Temporal difference learning is used to optimize the value network parameters and update the value evaluation by minimizing the loss function.

7. A low-latency and high-efficiency communication device for a switch, characterized in that: include: A processor for running an intelligent scheduler; A memory, connected to the processor, for storing the intelligent scheduling program; FPGA acceleration device, used to accelerate data plane processing; The intelligent scheduling program is implemented when executed by the processor: Status collection function, used to periodically obtain and cache switch operation status information; A scheduling decision function, which analyzes the operation status information based on a deep reinforcement learning network model and generates a scheduling strategy; The model optimization function optimizes the deep reinforcement learning network model based on the reward value of the execution feedback.

8. A low-latency and high-efficiency communication device for a switch according to claim 7, characterized in that: The FPGA acceleration device specifically includes: Multi-stage pipeline parsing circuit, used for parallel processing of parsing of input data packets, to achieve fast parsing and classification of messages; A reconfigurable parallel processing circuit, connected to the pipeline parsing circuit, having multiple parallel processing channels and a load balancing circuit between channels, for parallel forwarding of parsed data packets; A data plane acceleration circuit, connected to the reconfigurable parallel processing circuit, including a forwarding table lookup circuit and a hardware queue scheduling circuit, for accelerating forwarding decisions and queue scheduling of data packets; The real-time monitoring circuit is used to collect performance indicators and resource status such as data packet processing delay, queue length, channel utilization, etc., and output the monitoring data to the intelligent scheduling model through the status feedback circuit.

Images