Efficient multi-queue communication flow control system based on FPGA (Field Programmable Gate Array) and DDR (Double Data Rate)

Abstract

The invention discloses an efficient multi-queue communication flow control system based on FPGA + DDR, and the system comprises an enqueue management module which is used for determining an enqueue based on a network communication data packet of a preceding stage buffer, and writing the data in the enqueue into a DDR queue of a DDR storage module through a queue read-write control module; the dequeue management module is used for determining a dequeue queue in the DDR queue and reading data in the dequeue queue into a lower-level buffer through the queue read-write control module; the data flow control module is used for recording the number of sending packets and the number of bytes of a channel corresponding to each dequeue queue, and when sending exceeds a preset threshold value, an enable signal is generated, so that the DDR storage module stops outputting data, caches the data in the DDR queue and waits for signal sending. According to the system, the bandwidth utilization rate of the DDR is improved, and meanwhile, the multi-queue data flow is accurately controlled.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and more specifically to an efficient multi-queue communication flow control system based on FPGA+DDR. Background Art

[0002] At present, the network data communication system has a large traffic volume and a wide variety of services. When data exchange equipment (Ethernet switches, three-layer switches, etc.) receives data from multiple ports and needs to forward data to the same port, competition for data priority will occur. In order to ensure fair and orderly data transmission, services are generally divided into several queues according to priority and stored in external memory. Then, data is taken out from each queue of the external memory and sent according to certain rules. It is also necessary to support queue flow control to achieve fairness, traffic shaping and other purposes.

[0003] There are two ways to implement network data exchange equipment: general-purpose switching chip solution and FPGA solution. The use of general-purpose switching chips (such as Broadcom 56334, Centec 7132, etc.) can quickly realize the function of the equipment, but it is not flexible. The function of the equipment is the function provided by the general-purpose switching chip. The FPGA solution is the opposite of the general-purpose switching chip solution. It can realize flexible functional requirements, but the development cycle is longer.

[0004] FPGA has limited internal resources. To cope with large data usage scenarios, FPGA solutions usually need to use external storage DDR chips to expand memory resources, making data flow more flexible. Since DDR is half-duplex for reading and writing, only read or write operations can be performed at the same time. In order to facilitate the reading and writing of internal data packets and control the forwarding direction, the usual practice is to add a user-defined header before the original message. The custom header usually contains: input port number, output port number, priority, total length of the packet and other information. When writing data to the DDR chip, the output port number and priority are used as the queue channel, and the queue channel is also the high-order address of the DDR address bus. Through queue cache management, data is read out from DDR according to the interface empty and full status and queue flow control information, and the lower level detects the data flow information and performs flow control.

[0005] However, in the current network communication field, the existing storage and forwarding scheme based on FPGA (Field Programmable Gate Array) + DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory, i.e. synchronous dynamic random access memory) can realize the storage and forwarding of a large amount of data, but it generally has the problem of low bandwidth utilization, and the multi-queue current limiting strategy is also insufficient in terms of accuracy and flexibility. Specifically, frequent read-write switching will greatly reduce the read-write efficiency. The length of the network communication data packet is a random value within a certain range. The smaller the message length, the more frequent the DDR will jump, resulting in a lower bandwidth utilization of the DDR; the larger the message length, the larger the burst length of the DDR read data, which will cause the flow control to become inaccurate.

[0006] Therefore, how to accurately control the multi-queue data flow while improving the DDR bandwidth utilization is a problem that technical personnel in this field need to solve urgently. Summary of the invention

[0007] In view of the above problems, the present invention provides an efficient multi-queue communication flow control system based on FPGA+DDR to at least solve some of the technical problems mentioned in the above background technology.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] The present invention provides an efficient multi-queue communication flow control system based on FPGA+DDR, comprising: an entry management module, a queue read-write control module, an exit management module, a data flow control module and a DDR storage module;

[0010] The enqueue management module is used to determine the enqueue queue based on the network communication data packets buffered at the previous stage, and write the data in the enqueue queue into the DDR queue of the DDR storage module through the queue read-write control module;

[0011] The dequeue management module is used to determine the dequeue queue in the DDR queue, and read the data in the dequeue queue to the lower buffer through the queue read-write control module;

[0012] The data flow control module is used to record the number of sent packets and bytes of the channel corresponding to each dequeue queue. When the sending exceeds the preset threshold value, an enable signal is generated to stop the DDR storage module from outputting data, cache it in the DDR queue, and wait for the sending signal.

[0013] Further, the enqueue management module includes: a front-end buffer monitoring submodule, a first DDR state monitoring submodule and an enqueue queue parsing submodule;

[0014] The front-stage buffer monitoring submodule is used to monitor and obtain the queue data of the network communication data packets in the front-stage buffer;

[0015] The first DDR state monitoring submodule is used to monitor whether the current state of the DDR storage module is an accepting queue state;

[0016] The enqueue queue parsing submodule is used to determine the enqueue queue corresponding to the enqueue data according to the port number and the priority.

[0017] Further, the queue read / write control module includes: a DDR read / write state conversion control submodule, a DDR write command timing control submodule, a DDR write data timing control submodule, a DDR read command timing control submodule and a DDR read data timing control submodule;

[0018] The DDR read / write state conversion control submodule is used to control the data writing operation or data reading operation on the DDR storage module;

[0019] The DDR write command timing control submodule is used to provide a write command according to a first preset data length;

[0020] The DDR write data timing control submodule is used to write the data in the enqueue queue into the DDR storage module according to the write command;

[0021] The DDR read command timing control submodule is used to provide a read command according to the second preset data length;

[0022] The DDR read data timing control submodule is used to read the data in the DDR storage module into the lower buffer according to the read command.

[0023] Further, the dequeue management module includes: a second DDR status monitoring submodule, a dequeue queue parsing submodule and a post-buffer monitoring submodule;

[0024] The second DDR status monitoring submodule monitors whether there is dequeued data in the DDR queue of the DDR storage module;

[0025] The outgoing queue parsing submodule is used to determine the outgoing queue corresponding to the outgoing data according to the exit channel and length information;

[0026] The post-stage buffer monitoring submodule is used to monitor whether the current state of the post-stage buffer is the receiving dequeue state.

[0027] Furthermore, it also includes a time slice management module;

[0028] The time slice management module is used to manage the entry time of the entry queue and the exit time of the exit queue by presetting the read and write DDR cycle, ensuring that only the write DDR operation or the read DDR operation can be performed in the same time period.

[0029] Furthermore, the preset read and write DDR cycle includes:

[0030] If the current state is DDR write state and the front buffer of DDR is not empty, the DDR write state will be maintained until the front buffer is empty or exceeds the preset time, then it will be converted to DDR read state;

[0031] If the current state is DDR read, and there is always data in the DDR queue that needs to be dequeued, the DDR read state will continue until the dequeuing queue is empty or exceeds the preset time, then it will be converted to DDR write state.

[0032] Furthermore, it also includes a queue buffer management module;

[0033] The queue buffer management module is used to control the enqueue queue to perform an enqueue operation and control the dequeue queue to perform a dequeue operation.

[0034] Further, the queue buffer management module includes: an entry queue buffer submodule, an exit queue buffer submodule and a state determination module;

[0035] The enqueue queue cache submodule is used to store the read address, write address and queue depth of the enqueue queue;

[0036] The dequeue queue cache submodule is used to store the read address, write address and queue depth of the dequeue queue;

[0037] The state judgment module is used to judge the current queue state of the enqueue queue according to the read address, write address and queue depth of the enqueue queue; and to judge the current queue state of the outqueue queue according to the read address, write address and queue depth of the outqueue queue.

[0038] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a high-efficiency multi-queue communication flow control system based on FPGA+DDR, which has the following beneficial effects:

[0039] 1. The present invention can detect the flow information of each channel in real time through the data flow control module, flexibly control the flow of each channel, and use the DDR queue as a large cache to effectively deal with the situation of a large amount of data bursts, limit the data flow with the average bandwidth, and improve the DDR bandwidth utilization while accurately controlling the data flow of multiple queues.

[0040] 2. In the queue read / write control module of the present invention, the command channel and the data channel are relatively independent. This can be used to realize separate processing of commands and data. For example, when reading data from a DDR storage module, the read command can be sent first, and the command for the next read / write cycle can be sent without waiting for the data to be completely out. This makes full use of the time delay between reading the command and reading the valid data, thereby improving the overall bandwidth utilization of DDR.

[0041] 3. The present invention adopts a time slice management module, which allows continuous writing or reading of a large amount of data in the same time period, thereby increasing the time proportion of reading DDR state and writing DDR state in the entire state machine, reducing the switching frequency between read commands and write commands, reducing the switching frequency of read and write channel numbers, and especially increasing the time proportion of reading DDR state, thereby improving the overall DDR bandwidth utilization.

[0042] 4. Data packets are stored in DDR in the form of packets. There is no free address interval between packets in the same queue. Therefore, a specific custom packet interval is designed between packets. When a packet of data is written to the cache address, a custom packet interval data of an address is added to distinguish the end of the current packet and the start of the next packet. The DDR read and write pointer counts by byte stream, which can realize continuous reading and writing of byte streams. This method eliminates the fragmented space between data packets in the same cache channel and makes full use of the DDR cache capacity.

[0043] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0045] Figure 1 A schematic diagram of the framework of an efficient multi-queue communication flow control system based on FPGA+DDR provided in an embodiment of the present invention.

[0046] Figure 2 A schematic diagram of the principle framework of multiple queue entry and exit provided by an embodiment of the present invention.

[0047] Figure 3 A schematic diagram of the queue flow control principle framework provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1 As shown, the embodiment of the present invention discloses an efficient multi-queue communication flow control system based on FPGA+DDR, including an entry management module, a queue read-write control module, an exit management module, a data flow control module and a DDR storage module; wherein:

[0050] An enqueue management module is used to determine an enqueue queue based on the network communication data packets buffered at the previous stage, and write the data in the enqueue queue into the DDR queue of the DDR storage module through the queue read-write control module;

[0051] The dequeue management module is used to determine the dequeue queue in the DDR queue and read the data in the dequeue queue to the lower buffer through the queue read and write control module;

[0052] The data flow control module is used to record the number of packets and bytes sent by the channel corresponding to each dequeue queue. When the sending exceeds the preset threshold value, an enable signal is generated to stop the DDR storage module from outputting data, cache it in the DDR queue, and wait for the sending signal.

[0053] The system improves the DDR data cache efficiency, increases the flow control of the corresponding queue, and realizes the flow control of multi-port multi-priority queues while using the flexibility of FPGA. Next, each of the above modules is explained respectively.

[0054] 1. Team management module

[0055] The queue management module includes: a front-end buffer monitoring submodule, a first DDR state monitoring submodule and a queue analysis submodule; wherein:

[0056] (1) a front-stage buffer monitoring submodule, used to monitor and obtain the queue data of the network communication data packets in the front-stage buffer;

[0057] (2) a first DDR state monitoring submodule, used to monitor whether the current state of the DDR storage module is an accepting queue state;

[0058] (3) Queue parsing submodule, such as Figure 2 As shown, according to the port number and priority, the queue corresponding to the queued data is determined;

[0059] Then, the queue read and write control module writes the input in the queue into the address segment corresponding to the DDR, specifically, into the DDR queue of the DDR storage module, so that the data can be stored continuously in the DDR, thereby improving storage efficiency.

[0060] 2. Queue read and write control module

[0061] The queue read-write control module includes: a DDR read-write state conversion control submodule, a DDR write command timing control submodule, a DDR write data timing control submodule, a DDR read command timing control submodule and a DDR read data timing control submodule; by separating the read-write command from the read-write data, the DDR can be efficiently operated; specifically, wherein:

[0062] (1) A DDR read / write state conversion control submodule, which is used to control the writing or reading of data to the DDR storage module;

[0063] (2) a DDR write command timing control submodule, configured to provide a write command according to a first preset data length;

[0064] (3) a DDR write data timing control submodule, used to write the data in the queue into the DDR storage module according to the write command;

[0065] (4) a DDR read command timing control submodule, used to provide a read command according to a second preset data length;

[0066] (5) A DDR read data timing control submodule, which is used to read the data in the DDR storage module into the lower-level buffer according to the read command.

[0067] 3. Team management module

[0068] The dequeue management module includes: a second DDR status monitoring submodule, a dequeue queue parsing submodule and a post-buffer monitoring submodule; wherein:

[0069] (1) a second DDR status monitoring submodule, monitoring whether there is dequeued data in the DDR queue of the DDR storage module;

[0070] (2) a dequeue queue parsing submodule, used to determine the dequeue queue corresponding to the dequeue data according to the exit channel and length information;

[0071] (3) A back-end buffer monitoring submodule is used to monitor whether the current state of the back-end buffer is the receiving dequeue state.

[0072] In the specific operation process, the second DDR status monitoring submodule monitors whether each queue status can be dequeued, and combines whether the back-end buffer supports dequeuing. When sending, the dequeued data is synchronously parsed to obtain the corresponding exit and length information of the dequeued data, generate the corresponding channel number and read and write enable, and write it into the lower-level buffer; during the sending process, the data flow control module detects whether the flow control threshold is reached and uses the high-priority queue priority scheduling strategy to make the high-priority data dequeued first;

[0073] 4. Data flow control module

[0074] Data packets are organized in DDR and stored in packets. Data is written to DDR in packets. There is no free address interval between packets in the same queue. Therefore, a specific custom packet interval is designed between packets. When a packet of data is written to the cache address, a custom packet interval data of an address is added to distinguish the end of the current packet from the beginning of the next packet. When reading data from DDR, it is done in byte streams. Multiple data packets can be read at a time. The packet header information is determined by the interval between packets. Each packet header carries the channel number and packet length information; Figure 3 As shown, the out-of-queue data in the DDR storage module is accompanied by the channel number, enable, and data length information, which are collected by the data flow control module to record the flow information of the current channel in real time; the bandwidth of each channel can be parameterized to support the number of packets or bytes passed in 1ms to achieve bandwidth control. If the flow information of the current channel is not greater than the current limiting information, the transmission can continue; if the flow information of the current channel is greater than the preset threshold value, the preset threshold value is subtracted after the current detection, indicating the remaining flow information of the current channel, waiting for the next detection, and generating the flow control enable of this channel, so that the current channel no longer sends data;

[0075] When the dequeue queue is read out, the data is read out from the DDR and written to the lower-level cache area. The write channel number and data enable, along with the write end flag and the write data length, are used in the data flow control module; the data flow control module uses a flow controller, which is hung beside the DDR queue and does not affect the DDR output data. It can accurately control to pass N packets or M bytes of data in 1ms.

[0076] The embodiment of the present invention also includes:

[0077] 5. Time slice management module;

[0078] The time slice management module is used to manage the entry time of the entry queue and the exit time of the exit queue by presetting the DDR read and write cycle, ensuring that only DDR write operation or DDR read operation can be performed in the same time period; thereby improving the efficiency of continuous DDR read and write.

[0079] Among them, the preset read and write DDR cycle includes:

[0080] If the current state is DDR write state and the front buffer of DDR is not empty, the DDR write state will be maintained until the front buffer is empty or exceeds the preset time, then it will be converted to DDR read state;

[0081] If the current state is DDR read, and there is always data in the DDR queue that needs to be dequeued, the DDR read state will continue until the dequeuing queue is empty or exceeds the preset time, then it will be converted to DDR write state.

[0082] The embodiment of the present invention also includes:

[0083] 6. Queue cache management module;

[0084] The queue cache management module is used to control the queue entry operation and the queue exit operation. Specifically:

[0085] The queue buffer management module includes: an entry queue buffer submodule, an exit queue buffer submodule and a state judgment module; wherein:

[0086] (1) an enqueue queue cache submodule, used to store the read address, write address and queue depth of the enqueue queue;

[0087] (2) a dequeue queue cache submodule, used to store the read address, write address and queue depth of the dequeue queue;

[0088] (3) a state judgment module, used to judge the current queue state of the enqueue queue according to the read address, write address and queue depth of the enqueue queue; and used to judge the current queue state of the outqueue queue according to the read address, write address and queue depth of the outqueue queue.

[0089] In summary, an efficient multi-queue communication flow control system based on FPGA+DDR provided by an embodiment of the present invention, in the specific operation process, the network communication data packet is divided into queues by port number and priority in the queue management module, and is uniformly scheduled through shared memory, and different queue data is stored according to different address segments of DDR (i.e., DDR queues); it is continuously stored in the DDR storage module, and packet interval information is inserted between data packets, and it is performed according to byte stream when it is read from DDR; the operation of DDR is divided into read and write time slices, and only read or write operations are performed in one time slice; when reading data from DDR, the read command and data are separated and processed, and two situations are processed according to the queue depth; when reading data from DDR, the length information of each packet is accompanied, and the current flow of each out-queue queue is recorded by the data flow control module. Once the flow limit is exceeded, the full signal is fed back to control DDR to no longer send the current queue data. The present invention uses FPGA to improve the flexibility of scheduling, improve the effective bandwidth of DDR, make full use of the DDR cache capacity, and can achieve the purpose of flexible and accurate flow control of multiple queues.

[0090] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0091] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An efficient multi-queue communication flow control system based on FPGA+DDR, characterized in that: include: Enqueue management module, queue read and write control module, dequeue management module, data flow control module and DDR storage module; The enqueue management module is used to determine the enqueue queue based on the network communication data packets buffered at the previous stage, and write the data in the enqueue queue into the DDR queue of the DDR storage module through the queue read-write control module; The dequeue management module is used to determine the dequeue queue in the DDR queue, and read the data in the dequeue queue to the lower buffer through the queue read-write control module; The data flow control module is used to record the number of sent packets and bytes of the channel corresponding to each dequeue queue. When the sending exceeds the preset threshold value, an enable signal is generated to stop the DDR storage module from outputting data, cache it in the DDR queue, and wait for the sending signal.

2. According to the FPGA+DDR-based efficient multi-queue communication flow control system of claim 1, it is characterized in that: The queue management module includes: a front-end buffer monitoring submodule, a first DDR status monitoring submodule and a queue parsing submodule; The front-stage buffer monitoring submodule is used to monitor and obtain the queue data of the network communication data packets in the front-stage buffer; The first DDR state monitoring submodule is used to monitor whether the current state of the DDR storage module is an accepting queue state; The enqueue queue parsing submodule is used to determine the enqueue queue corresponding to the enqueue data according to the port number and the priority.

3. According to the FPGA+DDR-based efficient multi-queue communication flow control system of claim 1, it is characterized in that: The queue read and write control module includes: a DDR read and write state conversion control submodule, a DDR write command timing control submodule, a DDR write data timing control submodule, a DDR read command timing control submodule and a DDR read data timing control submodule; The DDR read / write state conversion control submodule is used to control the data writing operation or data reading operation on the DDR storage module; The DDR write command timing control submodule is used to provide a write command according to a first preset data length; The DDR write data timing control submodule is used to write the data in the enqueue queue into the DDR storage module according to the write command; The DDR read command timing control submodule is used to provide a read command according to the second preset data length; The DDR read data timing control submodule is used to read the data in the DDR storage module into the lower level buffer according to the read command.

4. The efficient multi-queue communication flow control system based on FPGA+DDR according to claim 1 is characterized in that: The dequeue management module includes: a second DDR status monitoring submodule, a dequeue queue parsing submodule and a post-buffer monitoring submodule; The second DDR status monitoring submodule monitors whether there is dequeued data in the DDR queue of the DDR storage module; The outgoing queue parsing submodule is used to determine the outgoing queue corresponding to the outgoing data according to the exit channel and length information; The post-stage buffer monitoring submodule is used to monitor whether the current state of the post-stage buffer is the receiving dequeue state.

5. The efficient multi-queue communication flow control system based on FPGA+DDR according to claim 1 is characterized in that: It also includes a time slice management module; The time slice management module is used to manage the entry time of the entry queue and the exit time of the exit queue by presetting the read and write DDR cycle, ensuring that only the write DDR operation or the read DDR operation can be performed in the same time period.

6. The efficient multi-queue communication flow control system based on FPGA+DDR according to claim 5, characterized in that: The preset read and write DDR cycle includes: If the current state is DDR write state and the front buffer of DDR is not empty, the DDR write state will be maintained until the front buffer is empty or exceeds the preset time, then it will be converted to DDR read state; If the current state is DDR read, and there is always data in the DDR queue that needs to be dequeued, the DDR read state will continue until the dequeuing queue is empty or exceeds the preset time, then it will be converted to DDR write state.

7. The efficient multi-queue communication flow control system based on FPGA+DDR according to claim 1, characterized in that: Also includes a queue cache management module; The queue buffer management module is used to control the enqueue queue to perform an enqueue operation and control the dequeue queue to perform a dequeue operation.

8. The efficient multi-queue communication flow control system based on FPGA+DDR according to claim 1, characterized in that: The queue buffer management module includes: an entry queue buffer submodule, an exit queue buffer submodule and a state judgment module; The enqueue queue cache submodule is used to store the read address, write address and queue depth of the enqueue queue; The dequeue queue cache submodule is used to store the read address, write address and queue depth of the dequeue queue; The state judgment module is used to judge the current queue state of the enqueue queue according to the read address, write address and queue depth of the enqueue queue; and to judge the current queue state of the outqueue queue according to the read address, write address and queue depth of the outqueue queue.

Images