Traffic output method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a traffic output method and device, electronic equipment and storage medium, and relates to the field of traffic control. The method comprises: obtaining a target data stream, and a bandwidth of hardware used for processing the target data stream and a number of physical ports; determining a number of logical ports according to the target data stream and the bandwidth; dividing the target data stream based on the number of logical ports to obtain a plurality of sub-data streams; and sequentially outputting processing results of the sub-data streams from the physical ports according to the number of physical ports and the number of logical ports. The present disclosure can solve the problem of mismatch between front-end traffic and back-end bandwidth without increasing hardware cost.

Description

Technical Field

[0001] This application relates to the field of communication technology, specifically to the field of flow control, and more particularly to a flow output method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of mobile data services and the internet, various network services are emerging one after another, resulting in ever-increasing network traffic. This can cause network bandwidth to become a bottleneck for some pre-deployed backend business systems. Because servers are only equipped with fixed-bandwidth network cards during deployment, the traffic that needs to be processed far exceeds the physical bandwidth of the network cards as traffic grows rapidly.

[0003] To address this issue, two common methods exist: 1. Increase the number of servers; 2. Replace the server's network interface cards (NICs) with those supporting higher bandwidth. For option 1, increasing hardware costs are prohibitively high. Furthermore, if only the NIC bandwidth is insufficient, and the CPU processing power is still sufficient, increasing the number of servers would waste CPU resources. It's also crucial to consider whether there's enough rack space in the data center to add more servers. If the racks are full, adding more servers is even less feasible. Therefore, this option has the lowest cost-effectiveness. For option 2, replacing the NICs with higher bandwidth ones only provides temporary relief. As data services continue to grow and network bandwidth increases, the servers in option 2 will eventually become a bottleneck again, and this solution doesn't fundamentally solve the problem. Moreover, high-bandwidth NICs are more expensive. After replacing the NICs, compatibility issues and insufficient CPU processing performance may arise between the server and the existing system, increasing the risk of system instability. Summary of the Invention

[0004] Embodiments of this disclosure provide a flow output method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a traffic output method, comprising: acquiring a target data stream and the bandwidth and number of physical ports of hardware used to process the target data stream; determining the number of logical ports based on the target data stream and the bandwidth; dividing the target data stream based on the number of logical ports to obtain multiple sub-data streams; and outputting the processing results of each sub-data stream from the physical ports in sequence according to the number of physical ports and the number of logical ports.

[0006] Secondly, embodiments of this disclosure provide a traffic output device, comprising: an acquisition unit configured to acquire a target data stream and the bandwidth and number of physical ports of hardware used to process the target data stream; a determination unit configured to determine the number of logical ports based on the target data stream and the bandwidth; a division unit configured to divide the target data stream based on the number of logical ports to obtain multiple sub-data streams; and an output unit configured to output the processing results of each sub-data stream sequentially from the physical ports based on the number of physical ports and the number of logical ports.

[0007] Thirdly, embodiments of this disclosure provide an electronic device including a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the flow output method as described in the first aspect.

[0008] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the flow output method as described in the first aspect.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 An exemplary system architecture diagram in which an embodiment of the flow output method of this disclosure can be applied;

[0012] Figure 2 This is a schematic flowchart of an embodiment of the flow output method of this disclosure;

[0013] Figure 3 This is a schematic flowchart of another embodiment of the flow output method of this disclosure;

[0014] Figure 4 This is a schematic diagram of the structure of one embodiment of the flow output device of this disclosure;

[0015] Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device disclosed herein. Detailed Implementation

[0016] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0017] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0018] Where there is no conflict, the embodiments and features described herein can be combined with each other.

[0019] To make the technical solutions and advantages of this disclosure clearer, the following description, in conjunction with the accompanying drawings and specific embodiments, will provide a more detailed account of this disclosure.

[0020] Figure 1 An exemplary system architecture 100 is shown that can be applied to embodiments of the flow output method or flow output device of this disclosure.

[0021] like Figure 1 As shown, the system architecture 100 may include a traffic splitter 101 and multiple servers 102, 103, and 104. The traffic splitter 101 and the multiple servers 102, 103, and 104 can interact via a network. The network may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0022] The distribution device 101 can distribute the received data packets to servers 102, 103, and 104. Each server 102, 103, and 104 can perform various processing on the data packets and output the processing results.

[0023] The distribution device 101 and the multiple servers 102, 103, and 104 can be either hardware or software. When the devices are hardware, they can be various electronic devices, including but not limited to servers, data centers, etc. When the devices are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module. No specific limitations are made here.

[0024] It should be noted that the flow output method provided in this embodiment is generally executed by the flow splitting device 101. Accordingly, the flow output device is generally disposed in the flow splitting device 101.

[0025] It should be understood that Figure 1 The number of distribution devices and servers shown is merely illustrative. Depending on implementation needs, any number of distribution devices and servers can be used.

[0026] Figure 2 A flow 200 of one embodiment of the flow output method of this disclosure is shown. For example... Figure 2 As shown, the flow output method of this embodiment may include the following steps:

[0027] Step 201: Obtain the target data stream and the bandwidth and number of physical ports of the hardware used to process the target data stream.

[0028] In this embodiment, the execution body of the flow output method (e.g.) Figure 1 The distribution device 101 shown can acquire the target data stream. The target data stream can be a data stream to be processed, which can be various data streams generated during the operation of the service. The executing entity can also acquire the bandwidth and the number of physical ports of the hardware network interface card used to process the data stream. Here, the physical ports can be the traffic output ports of the aforementioned hardware. In some specific practices, the aforementioned hardware may include multiple network interface cards, and the bandwidth of each network interface card may be the same or different.

[0029] Step 202: Determine the number of logical ports based on the target data stream and bandwidth.

[0030] In this embodiment, after obtaining the target data stream and bandwidth, the executing entity can calculate the theoretical number of ports required to process the target data stream based on the bandwidth. Specifically, the executing entity can directly divide the value of the target data stream by the bandwidth value, and the resulting value is the number of logical ports. Here, different logical ports have different identifiers.

[0031] Step 203: Divide the target data stream based on the number of logical ports to obtain multiple sub-data streams.

[0032] After obtaining the number of logical ports, the executing entity can divide the target data stream into multiple sub-data streams. Specifically, the executing entity can divide the target data stream equally, resulting in the same number of sub-data streams as the number of logical ports. Alternatively, the executing entity can divide the data stream according to its specific content, such as ensuring that each data stream contains the same data type.

[0033] Step 204: Based on the number of physical ports and the number of logical ports, output the processing results of each sub-data stream from the physical ports in sequence.

[0034] After obtaining the number of physical ports and logical ports, the executing entity can select a target port from these ports to output the processing results of each sub-data stream. Specifically, the executing entity can select a port with the same position identifier as the sub-data stream in the target data stream. Alternatively, the executing entity can randomly select a port from the ports as the target port.

[0035] The traffic output method provided by the above embodiments of this disclosure can process large traffic data streams without adding hardware, avoiding data stream drop-off caused by the mismatch between the large traffic data stream at the front end and the bandwidth of the back end hardware in existing solutions.

[0036] See Figure 3 This illustrates flow 300 of another embodiment of the flow output method according to this disclosure. Figure 3 As shown, the method in this embodiment may include the following steps:

[0037] Step 301: Obtain the target data stream and the bandwidth and number of physical ports of the hardware used to process the target data stream.

[0038] Step 302: Determine the number of logical ports based on the target data stream and bandwidth.

[0039] Step 303: Divide the target data stream based on the number of logical ports to obtain multiple sub-data streams.

[0040] Step 304: Determine the target port corresponding to each sub-data stream; in response to determining that the target port is a physical port, process the sub-data stream using hardware and output the processing result from the target port.

[0041] The executing entity can determine the corresponding target port for each sub-data stream. Specifically, the executing entity can use the identifier of the sub-data stream to select logical ports with the same identifier as the target ports. Alternatively, the executing entity can use a load balancing algorithm to determine a hash value for each sub-data stream. Then, the hash value is modulo the number of logical ports to obtain the identifier of the target port.

[0042] After determining the target port, the executing entity can determine whether the target port is a physical port. If it is, it can directly use hardware to process the sub-data stream and output the processing result from the target port.

[0043] Step 305: In response to determining that the target port is a logical port, the sub-data stream is discarded; the physical port is moved to the target port, the sub-data stream is reprocessed, and the processing result is output from the target port.

[0044] If the target port is not a physical port but a logical port, the executing entity can directly discard the sub-data stream. That is, the logical port in this scheme cannot output traffic. In this case, the executing entity can move the physical port to the target port. Since the sub-data stream is continuously sent, it can be reprocessed. Then, the processing result is output from the target port.

[0045] In some optional implementations of this embodiment, the hardware for processing the data stream includes registers of the switching chip. By modifying the contents of the registers, the physical port can be moved, thereby enabling polling output of traffic. The executing entity can start a monitoring thread for modifying the registers.

[0046] Step 306: In response to receiving a polling end command during the processing of the target data stream, determine whether the target data stream has been processed; in response to determining that the target data stream has not been processed, determine an interval time based on the sub-data stream and bandwidth, and when the interval time arrives, determine again whether the target data stream has been processed.

[0047] If a polling end command is received during the processing of the target data stream, it can be determined first whether the target data stream has been processed. Specifically, the executing entity can count the number of times the physical port outputs traffic; if the count is the same as the number of logical ports, the target data stream can be considered processed. Alternatively, the executing entity can determine whether processing is complete by querying a pre-set global identifier. This global identifier can include at least two values, with different values ​​representing different states. The executing entity can query the value of the global identifier; if it is the preset value, it indicates that processing is complete; otherwise, it indicates that processing is not complete.

[0048] If the target data stream has not been processed, an interval can be determined. This interval can be understood as the time required for the hardware to process a single sub-data stream. The executing entity can query whether the target data stream has been processed at the above interval.

[0049] In some optional implementations of this embodiment, if the target data stream is not fully processed, the executing entity can control the monitoring thread to sleep and set a timer. The duration of this timer is the same as the interval time. When the interval time is reached, it indicates that the sub-data stream has been processed, and the processing result of the sub-data stream can be output from the target port. Simultaneously, the monitoring thread can be woken up to further modify the registers and move the physical port. Alternatively, the monitoring thread can further query whether the target data stream has been fully processed.

[0050] Step 307: In response to the determination that the target data stream processing is complete, the monitoring thread ends.

[0051] If the target data stream has been processed, it means that no further modifications to the registers are needed, and the monitoring thread can be terminated.

[0052] The traffic output method provided in the above embodiments of this disclosure can divide the data stream according to the bandwidth of the backend device, so that the data stream can be fully processed by the backend device, and solve the problem of mismatch between frontend traffic and backend bandwidth without increasing hardware costs.

[0053] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a flow output device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0054] like Figure 4 As shown, the flow output device 400 of this embodiment includes: an acquisition unit 401, a determination unit 402, a division unit 403, and an output unit 404.

[0055] The acquisition unit 401 is configured to acquire the target data stream and the bandwidth and number of physical ports of the hardware used to process the target data stream.

[0056] The determining unit 402 is configured to determine the number of logical ports based on the target data stream and bandwidth.

[0057] The partitioning unit 403 is configured to partition the target data stream based on the number of logical ports to obtain multiple sub-data streams.

[0058] Output unit 404 is configured to output the processing results of each sub-data stream from the physical ports in sequence, based on the number of physical ports and the number of logical ports.

[0059] In addition, an electronic device is also proposed in the technical solution of this application.

[0060] Figure 5 A schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure is shown.

[0061] like Figure 5As shown, the electronic device may include a processor 501, a memory 502, a bus 503, and a computer program stored in the memory 502 and executable on the processor 501. The processor 501 and the memory 502 communicate with each other via the bus 503. When the processor 501 executes the computer program, it implements the steps of the above method, including, for example: acquiring a target data stream and the bandwidth and number of physical ports of the hardware used to process the target data stream; determining the number of logical ports based on the target data stream and the bandwidth; dividing the target data stream based on the number of logical ports to obtain multiple sub-data streams; and sequentially outputting the processing results of each sub-data stream from the physical ports according to the number of physical ports and the number of logical ports.

[0062] In addition, one embodiment of this disclosure also provides a non-transitory computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the above-described method, including, for example,: acquiring a target data stream and the bandwidth and number of physical ports of the hardware used to process the target data stream; determining the number of logical ports based on the target data stream and the bandwidth; dividing the target data stream based on the number of logical ports to obtain multiple sub-data streams; and outputting the processing results of each sub-data stream from the physical ports in sequence according to the number of physical ports and the number of logical ports.

[0063] In summary, the technical solution disclosed herein can divide the data stream according to the bandwidth of the backend device, thereby enabling the data stream to be fully processed by the backend device, and solving the problem of mismatch between frontend traffic and backend bandwidth without increasing hardware costs.

[0064] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A flow output method, comprising: Acquire the target data stream and the bandwidth and number of physical ports of the hardware used to process the target data stream; The number of logical ports is determined based on the target data stream and the bandwidth. The target data stream is divided into multiple sub-data streams based on the number of logical ports; Based on the number of physical ports and the number of logical ports, the processing results of each sub-data stream are output from the physical ports in sequence.

2. The method according to claim 1, wherein, The step of sequentially outputting the processing results of each sub-data stream from the physical port according to the number of physical ports and the number of logical ports includes: Determine the target port corresponding to each sub-data stream; In response to determining that the target port is a physical port, the hardware processes the sub-data stream and outputs the processing result from the target port.

3. The method according to claim 1, wherein, The hardware includes the registers of the switching chip; as well as The step of sequentially outputting the processing results of each sub-data stream from the physical port according to the number of physical ports and the number of logical ports includes: Determine the target port corresponding to each sub-data stream; In response to determining that the target port is a logical port, the sub-data stream is discarded; Start the monitoring thread; The monitoring thread is controlled to modify the register, change the target port to a physical port, continue to receive sub-data streams through the physical port, reprocess the sub-data streams, and output the processing result from the target port.

4. The method according to claim 1, wherein, Determining the target port corresponding to each sub-data stream includes: The hash value is determined for each sub-data stream based on the load balancing algorithm; The target port is determined based on the hash value and the number of logical ports.

5. The method according to claim 3, wherein, The method further includes: In response to receiving a polling end command during the processing of the target data stream, determine whether the target data stream has been processed. In response to determining that the target data stream has not been processed, an interval time is determined based on the sub-data stream and the bandwidth, and when the interval time is reached, it is determined again whether the target data stream has been processed.

6. The method according to claim 5, wherein, Determining whether the target data stream has been processed includes: The control monitoring thread queries a pre-set global identifier to determine whether the target data stream has been processed.

7. The method according to claim 6, wherein, The step of responding to determining that the target data stream has not been processed, determining an interval time based on the sub-data stream and the bandwidth, and determining again whether the target data stream has been processed when the interval time is reached, includes: In response to determining that the target data stream has not been fully processed, the monitoring thread is controlled to sleep; Set a timer according to the specified interval; When the timer expires, the monitoring thread is woken up and the global identifier is queried again.

8. The method according to claim 5, wherein, The method further includes: Upon determining that the target data stream processing is complete, the monitoring thread terminates.

9. A flow output device, comprising: The acquisition unit is configured to acquire the target data stream and the bandwidth and number of physical ports of the hardware used to process the target data stream; The determining unit is configured to determine the number of logical ports based on the target data stream and the bandwidth; The partitioning unit is configured to divide the target data stream based on the number of logical ports to obtain multiple sub-data streams; The output unit is configured to output the processing results of each sub-data stream from the physical port in sequence, according to the number of physical ports and the number of logical ports.

10. An electronic device comprising a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the flow output method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the flow output method as described in any one of claims 1 to 8.

Images