Burst traffic processing method and system for multi-stage large cache

Abstract

The invention relates to the technical field of cables, in particular to a burst flow processing method and system for a multi-stage large cache, and discloses a burst flow processing system for the multi-stage large cache, which comprises a first-stage cache module and a second-stage cache module, the first-stage cache module internally comprises a first-stage cache flow control module, the second-stage cache module internally comprises a second-stage cache flow control module, and the second-stage cache flow control module internally comprises a second-stage cache flow control module. The interface configuration information is obtained through the first-level cache module, if the second-level cache is needed for configuring the current interface, the flow is sent to the second-level cache flow control module, and if the direct output of the current interface is configured and the second-level cache is not needed, the flow is directly sent to the corresponding interface for output through exchange. According to the method, the first-level cache and the second-level cache are matched, the cache bandwidth and the cache capacity are greatly expanded, the corresponding flow control threshold value can be configured according to the final output bandwidth, and therefore zero-packet-loss operation of a multi-link large burst convergence scene is achieved.

Description

Technical Field

[0001] The invention relates to the field of Ethernet optical fiber communication, and in particular to a method and system for processing burst traffic of a multi-level large cache. Background Art

[0002] In application scenarios in certain fields, such as the financial field, there are usually a large number of splitter collection nodes. Although the average operating traffic of each splitter node is not high, the instantaneous burst traffic is high. For example, the market synchronization at the opening time of securities trading and the internal synchronization of the database of banking business usually cause millisecond-level ultra-high bandwidth bursts. The interface bandwidth of the final analysis system deployed at the back end is not high, and the collection and diversion equipment is required to perform aggregation operations with a large convergence ratio. This puts extremely high demands on the cache anti-burst performance of the diversion equipment when converging multi-link traffic.

[0003] The port convergence in the application scenarios of the financial field is relatively high. Therefore, the part of the total access traffic that exceeds the output bandwidth must be absorbed by the internal cache of the device. However, the traditional collection and aggregation device architecture has limited internal message cache, and generally the larger one can only reach about 36MB. When there is excessive micro-burst traffic, if the burst exceeds 10Gbps, the device can only absorb 28ms of excess traffic. When the burst exceeds 50Gbps, the device can only continue to receive 5ms of excess traffic. Usually, micro-bursts may reach the order of several hundred milliseconds. When the device cannot absorb the full amount, it will eventually lead to congestion and packet loss. The generation of packet loss greatly affects the reliability and credibility of the entire network collection. Summary of the invention

[0004] The present invention aims at the deficiencies in the prior art and provides the following technical solutions: A burst traffic processing system with a multi-level large cache includes: a first-level cache module and a second-level cache module.

[0005] Specifically, the first-level cache module includes a first-level cache flow control module, which obtains interface configuration information through the first-level cache module. If the current interface is configured to do second-level caching, the traffic is sent to the first-level cache flow control module. If the current interface is configured to output directly without the need for second-level caching, the traffic is directly sent to the corresponding interface output through switching.

[0006] As an improvement of the above technical solution, the first-level cache module includes a packet switching network module, a first-level cache flow control module, and an interconnection port transceiver module; the second-level cache module includes an interconnection port transceiver module, a message cache write module, a DDR array control module, a message cache read module, and an output flow control module; the packet switching network module is used to forward and exchange the traffic accessed by the Ethernet interface according to the rule configuration; the first-level cache flow control module is used to forward the received traffic to the interconnection port, and determine whether to enable the first-level cache according to the current traffic bandwidth situation; the interconnection port transceiver module is responsible for receiving and sending message traffic of several interconnection ports; the message cache write module is used to mark the accessed message traffic with a sequence number, generate a write command and distribute it to the DDR array control module; the message cache read module is used to generate a read command to read the traffic cached in the DDR array, and at the same time perform output sorting according to the sequence number mark of the message; the output flow control module is used to control the output speed of the traffic read from the DDR array, and control the bandwidth size of the output traffic not to exceed the configured threshold.

[0007] The method for processing burst traffic of a multi-level large cache is applied to the burst traffic processing system of a multi-level large cache as described in the above technical solution, and comprises the following steps: S10: The packet switching network module forwards and switches the traffic connected to the Ethernet interface according to the rule configuration.

[0008] S20: forwarding the received traffic to the interconnection port transceiver module through the first-level cache flow control module.

[0009] S30: Packing the message traffic received by the interconnection port transceiver module into a DDR write command through the message cache write module and sending it to the DDR array control module.

[0010] S40: Generate a read command through the message cache read module to read out the data flow cached by the DDR array, sort the output of the flow according to the sequence number label, and transfer the restored data packet to the output flow control module.

[0011] S50: The flow read out from the DDR array is output at a controlled rate according to the configured bandwidth threshold through the output flow control module to avoid subsequent congestion.

[0012] As an improvement of the above technical solution, step S10 includes the following steps: S11: The packet switching network module forwards the input traffic to any interface according to the configuration.

[0013] S12: Obtain interface configuration information. If the current interface is configured to be a secondary cache, the traffic is sent to the secondary cache flow control module. If the current interface is configured to be directly output without a secondary cache, the traffic is directly sent to the corresponding interface output through switching.

[0014] As an improvement of the above technical solution, when the traffic forwarded to the interconnection port in step S20 exceeds a threshold, the excess aggregated traffic is sent to the switching fast memory for caching.

[0015] As an improvement of the above technical solution, the DDR array control module is peripherally mounted with four groups of 64-bit DDR memories, and the access and reading of each DDR memory is individually controlled. The message traffic accessing the DDR memory is marked with a serial number, and after the write command is generated, it is distributed to the DDR array control module.

[0016] As an improvement of the above technical solution, the output interface of the output flow control module is deployed with a fixed cache of 64KB. After a whole packet is output, the water volume counter is accumulated, and then a fixed leakage value is subtracted every 5ns. When the entire counter is reduced to 0, the next message is allowed to continue to be output. The size of the output flow can be controlled by adjusting the size of the leakage value.

[0017] As an improvement of the above technical solution, the first-level cache module includes a switching chip, and the second-level cache module is provided with a second-level cache control chip, and the type of the second-level cache control chip is an ASIC chip.

[0018] Beneficial effects of the present invention: The first and second level caches are used to cooperate, which greatly expands the cache bandwidth and cache capacity, and the corresponding flow control threshold can be configured according to the final output bandwidth, so as to achieve zero packet loss operation in multi-link large burst aggregation scenarios. In addition, the DDR array control module with four-way parallel read and write organization is used to pre-mark the message data with sequence numbers and import them into the array cache in a round-robin manner, and then the read side re-sorts them by sequence numbers to ensure the strict restoration of the order of data packets and prevent disorder. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the architecture of the present invention; Figure 2 Flow diagram of the traditional cache solution and the multi-level cache solution of the present invention during execution. DETAILED DESCRIPTION

[0020] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0021] The port convergence in the application scenarios of the financial field is relatively high. Therefore, the part of the total access traffic that exceeds the output bandwidth must be absorbed by the internal cache of the device. However, the traditional collection and aggregation device architecture has limited internal message cache, and generally the larger one can only reach about 36MB. When there is excessive micro-burst traffic, if the burst exceeds 10Gbps, the device can only absorb 28ms of excess traffic. When the burst exceeds 50Gbps, the device can only continue to receive 5ms of excess traffic. Usually, micro-bursts may reach the order of several hundred milliseconds. When the device cannot absorb the full amount, it will eventually lead to congestion and packet loss. The generation of packet loss greatly affects the reliability and credibility of the entire network collection.

[0022] In order to solve the above problems, the following embodiments are provided: Example

[0023] A burst traffic processing system with a multi-level large cache includes: a first-level cache module and a second-level cache module.

[0024] Specifically, the first-level cache module includes a first-level cache flow control module, which obtains interface configuration information through the first-level cache module. If the current interface is configured to do second-level caching, the traffic is sent to the second-level cache flow control module. If the current interface is configured to output directly without the need for second-level caching, the traffic is directly sent to the corresponding interface output through switching.

[0025] In one embodiment, the first-level cache module includes a packet switching network module, a first-level cache flow control module interconnect port transceiver module, the second-level cache module includes an interconnect port transceiver module, a message cache write module, a DDR array control module, a message cache read module, and an output flow control module. The packet switching network module is used to forward and switch the traffic accessed by the Ethernet interface according to the rule configuration.

[0026] In order to solve the packet loss problem of burst traffic buffer aggregation under large convergence ratio, such as Figure 1 As shown, this embodiment adopts a multi-level large cache convergence solution. On the basis of the original switch chip's first-level cache, an ASIC chip is externally mounted as a second-level cache control chip. The two are interconnected using a 4x100G interface. The ASIC chip is externally mounted with a DDR storage array with a maximum capacity of up to 32GB for traffic cache. In addition to controlling the traffic access of the second-level cache, the ASIC can also control the output traffic bandwidth to make the final output smoother and avoid packet loss.

[0027] Among them, the first-level cache flow control module is used to forward the received traffic to the interconnection port, and determine whether to enable the second-level cache according to the current traffic bandwidth situation. The interconnection port transceiver module is responsible for receiving and sending message traffic of several interconnection ports. The message cache write module is used to mark the accessed message traffic with a serial number, generate a write command and distribute it to the DDR array control module. The message cache read module is used to generate a read command to read the traffic cached in the DDR array, and at the same time, output sorting is performed according to the serial number mark of the message. The output flow control module is used to control the output speed of the traffic read from the DDR array, and control the bandwidth size of the output traffic not to exceed the configured threshold. Example

[0028] In order to cooperate with the first embodiment, a method for processing burst traffic of a multi-level large cache is also provided, which is applied to the burst traffic processing system of the multi-level large cache described in the first embodiment, and includes the following steps: S10: The packet switching network module forwards and switches the traffic connected to the Ethernet interface according to the rule configuration.

[0029] Specifically, step S10 includes the following steps: S11: The packet switching network module forwards the input traffic to any interface according to the configuration.

[0030] This module is the internal packet switching network of the switching chip, which can forward the input traffic to any interface according to the configuration.

[0031] S12: Obtain interface configuration information. If the current interface is configured to have a secondary cache, the traffic is sent to the primary cache flow control module. If the current interface is configured to output directly without a secondary cache, the traffic is directly sent to the corresponding interface output through switching.

[0032] S20: forwarding the received traffic to the interconnection port transceiver module through the first-level cache flow control module.

[0033] After receiving the traffic, the module forwards it to the interconnection port. Since the internal interconnection port has only 400Gbps bandwidth capacity, if the current aggregated burst traffic that needs to be forwarded to the interconnection port exceeds 400Gbps, the excess aggregated traffic is sent to the switch level 1 fast memory for cache. If it does not exceed 400Gbps, it is directly sent out without the fast memory participating in the cache. Among them, for the interconnection port transceiver module, it is responsible for sending and receiving the message traffic of the four 100G interconnection ports.

[0034] S30: Packing the message traffic received by the interconnection port transceiver module into a DDR write command through the message cache write module and sending it to the DDR array control module.

[0035] The DDR array control module is peripherally mounted with four groups of 64-bit DDR memories, and the access and reading of each DDR memory is individually controlled. The message traffic accessing the DDR memory is marked with a serial number, and after the write command is generated, it is distributed to the DDR array control module.

[0036] S40: Generate a read command through the message cache read module to read out the data flow cached by the DDR array, sort the output of the flow according to the sequence number label, and transfer the restored data packet to the output flow control module.

[0037] S50: The flow read out from the DDR array is output at a controlled rate according to the configured bandwidth threshold through the output flow control module to avoid subsequent congestion.

[0038] Specifically, the output interface of the output flow control module is deployed with a fixed cache of 64KB. After a whole packet is output, the water volume counter is accumulated, and then a fixed leakage value is subtracted every 5ns. When the entire counter is reduced to 0, the next message is allowed to continue to be output. The size of the output flow can be controlled by adjusting the size of the leakage value.

[0039] The entire data flow workflow is as follows: traffic is accessed by the input interface of the device. Since it is the aggregation of multiple interfaces, the entire burst traffic is superimposed. When the function of the configured interface secondary cache module is turned on and a traffic micro-burst occurs on the current interface, the primary cache module first makes a bandwidth judgment. When it exceeds 400Gbps, the additional bandwidth starts the primary cache function and is absorbed by the cache (36MB) inside the switching chip. If the bandwidth does not exceed 400Gbps, it is directly forwarded to the interconnection port module. After the traffic is forwarded to the secondary cache module, all traffic is cached in the DDR array control module (32GB), and the traffic is output according to the currently configured output traffic speed control threshold. For example, when the total bandwidth of the final load balancing output is only 4 10G ports, the configured speed control threshold is 40Gbps. When the input traffic micro-burst exceeds 40Gbps, the output flow control module will back-pressure the message cache reading module to prevent the subsequent traffic from being read out. In this way, in the equipment installation scenario, the peak burst traffic caused by the superposition of multi-port converged traffic will be absorbed by the secondary large cache and smoothly controlled to 40Gbps output, avoiding packet loss.

[0040] In order to further test the solution described in this embodiment, the following test case is provided: Test conditions: 7 100G ports access, 4 10G interfaces output, each 100G interface access 4Gbps, total input traffic can reach 28Gbps, load balanced to 4 10G interfaces output, another 100G interface receives 1,000,000 random packets burst traffic at full bandwidth, the maximum burst traffic at this time will reach 128Gbps, and forwarding test is performed, the large cache function of the device is turned on and the output rate is limited to 40Gbps, the packet loss rate of the test result is 0%. Figure 2 As shown, Figure 2 The upper middle curve is the flow rate when the traditional solution is executed, and the lower curve is the flow rate when this embodiment is executed.

[0041] The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them. Anyone familiar with the technology can modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A burst traffic processing system with multi-level large cache, characterized in that: include: A first-level cache module, wherein the first-level cache module includes a first-level cache flow control module; Second level cache module; The interface configuration information is obtained through the first-level cache module. If the current interface is configured to be a second-level cache, the traffic is sent to the second-level cache flow control module. If the current interface is configured to output directly without the need for a second-level cache, the traffic is directly sent to the corresponding interface output through switching.

2. The burst traffic processing system with multi-level large cache according to claim 1 is characterized in that: The first-level cache module includes a packet switching network module and a first-level cache flow control module interconnection port transceiver module; The secondary cache module includes an interconnection port transceiver module, a message cache write module, a DDR array control module, a message cache read module, and an output flow control module; The packet switching network module is used to forward and switch the traffic accessed by the Ethernet interface according to the rule configuration; The first-level cache flow control module is used to forward the received traffic to the interconnection port and determine whether to enable the first-level cache according to the current traffic bandwidth; The interconnection port transceiver module is responsible for sending and receiving message traffic of several interconnection ports; The message cache write module is used to mark the incoming message traffic with a sequence number, generate a write command and distribute it to the DDR array control module; The message cache read module is used to generate a read command to read the traffic cached in the DDR array, and to sort the output according to the sequence number tag of the message; The output flow control module is used to control the output speed of the flow read from the DDR array, and control the bandwidth of the output flow not to exceed the configured threshold.

3. A method for processing burst traffic of a multi-level large cache, applied to a system for processing burst traffic of a multi-level large cache as claimed in claim 2, characterized in that: The steps include: S10: forwarding and switching the traffic connected to the Ethernet interface according to the rule configuration through the packet switching network module; S20: forwarding the received traffic to the interconnection port transceiver module through the first-level cache flow control module; S30: Packing the message traffic received by the interconnection port transceiver module into a DDR write command and sending it to the DDR array control module through the message cache write module; S40: Generate a read command through the message cache read module to read out the data flow cached by the DDR array, sort the output of the flow according to the sequence number label, and transfer the restored data packet to the output flow control module; S50: The flow read out from the DDR array is output at a controlled rate according to the configured bandwidth threshold through the output flow control module to avoid subsequent congestion.

4. The method for processing burst traffic of a multi-level large cache according to claim 3 is characterized in that: The step S10 comprises the following steps: S11: The packet switching network module forwards the input traffic to any interface according to the configuration; S12: Obtain interface configuration information. If the current interface is configured to be a secondary cache, the traffic is sent to the secondary cache flow control module. If the current interface is configured to be directly output without a secondary cache, the traffic is directly sent to the corresponding interface output through switching.

5. The method for processing burst traffic of a multi-level large cache according to claim 3 is characterized in that: When the traffic forwarded to the interconnection port in step S20 exceeds a threshold, the excess aggregated traffic is sent to the switching fast memory for caching.

6. The method for processing burst traffic of a multi-level large cache according to claim 3, characterized in that: The DDR array control module is peripherally mounted with four groups of 64-bit DDR memories, and the access and reading of each DDR memory is individually controlled. The message traffic accessing the DDR memory is marked with a serial number, and after the write command is generated, it is distributed to the DDR array control module.

7. The method for processing burst traffic of a multi-level large cache according to claim 3, characterized in that: The output interface of the output flow control module is deployed with a fixed cache of 64KB. After a whole packet is output, the water volume counter is accumulated, and then a fixed leakage value is subtracted every 5ns. When the entire counter is reduced to 0, the next message is allowed to continue to be output. The size of the output flow can be controlled by adjusting the size of the leakage value.

8. The method for processing burst traffic of a multi-level large cache according to any one of claims 3 to 7, characterized in that: The first-level cache module includes a switching chip, and the second-level cache module is provided with a second-level cache control chip, and the type of the second-level cache control chip is an ASIC chip.

Images