A multi-link parallel network communication method and apparatus

Abstract

This invention discloses a multi-link parallel network communication method and apparatus, belonging to the field of communication technology. The method fills each original network packet corresponding to each service flow with identification information to obtain the target network packet corresponding to each original network packet. The identification information includes: the total sequence number of the packet, the original length of the packet, the original protocol information of the packet, and the packet segmentation information. If acting as the sending side, multiple links are selected for parallel transmission of each target network packet. If acting as the receiving side, the target network packets are restored to their original, sequential order based on the identification information in each target network packet, and submitted to the upper layer to obtain the service flow, thereby achieving parallel transmission and reception of multiple network packets. This invention implements parallel transmission and reception at the IP layer, and without changing the original protocol mechanism of the network packets, it can more fully utilize the useful bandwidth of each link to obtain a higher aggregation throughput.

Description

Technical Field

[0001] This invention belongs to the field of communication technology, and more specifically, relates to a multi-link parallel network communication method and apparatus. Background Technology

[0002] With the rapid development of communication technology and the widespread coverage of mobile network infrastructure, network transmission quality is constantly improving, and more and more services are accessing the network. This leads to a continuous increase in the demand for throughput and reliability of communication systems. Faced with these growing user needs, further research on current communication systems is urgently required.

[0003] Today, most terminal devices can often use multiple wireless links, such as Wi-Fi and mobile data on different frequency bands, but in reality, users often only use one link for communication. Traditional single-link bandwidth is limited, and when encountering adverse environments that affect link transmission quality or when users need to transmit a large number of data packets, it can lead to packet loss, bit errors, increased latency, and other issues in the network, reducing user satisfaction. Summary of the Invention

[0004] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a multi-link parallel network communication method and apparatus. Its purpose is to fill each network packet corresponding to each service flow with identification information. If the sender is acting as a transmitter, multiple filled network packets can be transmitted in parallel. If the receiver is acting as a receiver, the data flow order can be restored based on the identification information in each network packet, thereby achieving parallel transmission and reception of multiple network packets. This solves the technical problem of low single-link throughput in the existing IP network layer.

[0005] To achieve the above objectives, according to one aspect of the present invention, a multi-link parallel network communication method is provided, comprising:

[0006] S1: Create send and receive message functions at the IP layer hook point;

[0007] S2: Fill identification information into each original network packet corresponding to each service flow to obtain the target network packet corresponding to each original network packet; the identification information includes: total sequence number of the packet, original length of the packet, original protocol information of the packet and packet segmentation information;

[0008] S3: When sending multiple target network packets corresponding to the service flow, select multiple links and use the packet sending function to transmit each target network packet in parallel;

[0009] S4: When multiple target network packets are received using the received packet function, each target network packet is restored into the original network packets arranged in sequence according to the identification information in each target network packet, and submitted to the upper layer so that it can obtain the service flow.

[0010] In one embodiment, S2 includes: filling the end of each original network packet corresponding to the service flow with identification information to obtain the target network packet corresponding to each original network packet.

[0011] In one embodiment, the identification information further includes: user sequence number and link sequence number.

[0012] In one embodiment, step S2, obtaining the target network packet corresponding to each of the original network packets, includes:

[0013] Determine whether the padded original network packet exceeds the system's maximum transmission unit;

[0014] If the limit is not exceeded, the original network packet after padding will be used as its corresponding target network packet;

[0015] If the number of packets exceeds the limit, the original network packet after padding will be split into multiple packets, and each packet will be used as its corresponding target network packet.

[0016] In one embodiment, S4 includes:

[0017] When multiple target network packets are received using the received packet function, they are sorted according to the identification information of each target network packet.

[0018] Determine whether each of the sorted target network packets is a segmented packet based on the packet segmentation information in the packet segmentation information.

[0019] If so, the original network packet is obtained by reassembling it according to the packet segmentation information and deleting its identification information; otherwise, the original network packet is obtained directly from its identification information.

[0020] Finally, it is submitted to the upper layer so that it can obtain the business flow.

[0021] In one embodiment, S3 includes: when sending multiple target network packets corresponding to the service flow,

[0022] S31: Create multiple ICMP link probe packets and send them to the receiving side in parallel through multiple links, and obtain the link attribute information of each parallel link based on the feedback message from the receiving side;

[0023] S32: Select multiple matching links based on the link attribute information of each link, and use the message sending function to send each of the target network messages in parallel.

[0024] In one embodiment, S31 includes:

[0025] On the sending side, an ICMP link probe packet is created for each link. The ICMP link probe packet carries information including: the packet loss rate for a certain link, the number of bytes received per unit time, the number of bytes sent per unit time, the control information from the sending side to the receiving side, and the message sending timestamp information. Each ICMP link probe packet is sent to the receiving side through the corresponding link, so that the receiving side parses the ICMP link probe packet for each link, retains the information from the sending side for local use, fills in the local confirmation of the number of bytes received per unit time, the number of bytes sent per unit time, and the control information from the sending side to the sending side, and returns it using the automatic ICMP message return mechanism.

[0026] After receiving the return message, the sending side calculates the link attribute information of each link using the sending timestamp and the local timestamp, and retains the information filled in by the receiving side for local use.

[0027] In one embodiment, the ICMP link probe packet carries information including: user ID number, link ID number, link transmission valid flag bit, control command valid flag bit, reserved bit, link probe message flag bit, packet classification, link delay of the last probe, total amount of local link transmission in the last probe cycle, total amount of local link reception in the last probe cycle, pointer structure for recording time, and local absolute time of sending probe request packet.

[0028] In one embodiment, S32 includes: the sending side selecting links that match each of the target network packets based on the link attribute information of all links, filling protocol information based on the link attribute information of the links that match each of the target network packets, and using the packet sending function to complete the packet sending through the protocol information; the protocol information includes IP address and port information.

[0029] According to another aspect of the present invention, a multi-link parallel network communication device is provided, comprising:

[0030] Create a module to create send and receive message functions at IP layer hook points;

[0031] The acquisition module is used to fill identification information into each original network packet corresponding to each service flow in order to obtain the target network packet corresponding to each original network packet; the identification information includes: total sequence number of the packet, original length of the packet, original protocol information of the packet and packet segmentation information.

[0032] The sending module is configured to select multiple links and use the sending message function to transmit each target network message in parallel when sending multiple target network messages corresponding to the service flow;

[0033] The receiving module is configured to, when receiving multiple target network packets using the receiving message function, restore each target network packet into the original network packets arranged in sequence according to the identification information in each target network packet, and submit them to the upper layer so that it can obtain the service flow.

[0034] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0035] (1) This invention provides a multi-link parallel network communication method. Identification information is filled into each network packet corresponding to each service flow. If acting as a sender, multiple filled network packets can be transmitted in parallel. If acting as a receiver, the data flow order can be restored according to the identification information in each network packet, thereby realizing the parallel sending and receiving of multiple network packets. Parallel sending and receiving transmission functions are implemented at the IP layer. Without changing the original protocol mechanism of network packets, the effective bandwidth of each link can be more fully utilized to obtain a larger aggregation throughput.

[0036] (2) This scheme inserts the identification information into the end of the network packet, which can reduce the number of times the network packet structure is copied in memory and moved by pointers in the LINUX kernel, thereby effectively improving the CPU utilization and greatly improving the efficiency of the parallel system in processing network packets.

[0037] (3) The identification information mentioned in this scheme also includes: user sequence number and link sequence number. When the receiving side receives the target network packet, it can quickly parse the order of the network packet, thereby reducing the computational complexity of the parsing process and improving communication efficiency.

[0038] (4) This scheme determines whether the original network packet after padding exceeds the maximum transmission unit (MTU) of the system. When the network packet created by the upper layer service is larger than the MTU after inserting the identification information, the service transmission can be completed by dividing it into multiple sub-packets that meet the MTU conditions.

[0039] (5) In this scheme, when the network packet created by the upper layer service is larger than the MTU after inserting the identification information, the packet is segmented. The packet segmentation information in each target network packet is used to determine whether it is a segmented packet. This can more accurately parse the order of network packets.

[0040] (6) This solution creates multiple ICMP link probe packets and sends them to the receiving side in parallel through multiple links, which can achieve more effective link probe function. By utilizing the mechanism of automatic return of ICMP message requests, ICMP link probe messages are created and sent at the sending terminal. The sending and receiving terminals complete the data interaction by filling attribute information in the message payload area.

[0041] (7) Both parties in this scheme can obtain each other's attribute information, which makes it easier to match the optimal message sending link and also enables the transmission of control information (link on / off, weight) between the two parties.

[0042] (8) The information carried by the ICMP link probe packet in this scheme includes: the link delay of the last probe, the total number of local links sent in the last probe cycle, the total number of local links received in the last probe cycle, a pointer structure for recording time, and the local absolute time of sending the probe request packet; it can quickly parse the link attribute information of each link, which is convenient for sending multiple target network packets in parallel.

[0043] (9) The protocol information described in this scheme includes IP address and port information to facilitate faster sorting and reassembly of received network packets by the receiving side. Attached Figure Description

[0044] Figure 1 This is a flowchart of the multi-link parallel network communication method provided in Embodiment 1 of the present invention.

[0045] Figure 2 This is a schematic diagram of the multi-link parallel network communication method provided in Embodiment 1 of the present invention.

[0046] Figure 3 This is a schematic diagram of the identification information in the multi-link parallel network communication method provided in Embodiment 3 of the present invention.

[0047] Figure 4 This is a schematic diagram of the ICMP link probe packet in the multi-link parallel network communication method provided in Embodiment 8 of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Example 1

[0050] like Figure 1 and Figure 2As shown, this embodiment provides a multi-link parallel network communication method, including: S1: creating a send message function and a receive message function at the IP layer HOOK point; S2: filling identification information into each original network packet corresponding to each service flow to obtain the target network packet corresponding to each original network packet; the identification information includes: total sequence number of the packet, original length of the packet, original protocol information of the packet, and packet segmentation information; S3: when sending multiple target network packets corresponding to the service flow, selecting multiple links to transmit each target network packet in parallel using the send message function; S4: when receiving multiple target network packets using the receive message function, restoring each target network packet into an ordered original network packet according to the identification information in each target network packet, and submitting it to the upper layer to obtain the service flow.

[0051] Specifically, this embodiment employs encapsulation technology that inserts identification information into network packets to enable service flows to be sent, aggregated, ordered, and submitted across multiple physical links. On the sending side, after intercepting network packets at the IP layer of the Linux kernel, it fills the network packets with identification information such as packet sequence number, link sequence number, and segmentation flag, and selects the optimal link for transmission based on the attribute information of each physical link. Figure 1 As shown, the receiving side can use the identification information in the network packets to place network packets received from multiple physical links into the receiving window, completing the work of network packet sorting, reassembly, and submission to the upper-layer protocol stack, ensuring that network service flows can achieve parallel transmission through this technology.

[0052] Example 2

[0053] In this embodiment, S2 includes: filling the end of each original network packet corresponding to the service flow with identification information to obtain the target network packet corresponding to each original network packet.

[0054] To ensure the stability and reliability of application-layer services while achieving efficient converged communication, parallel transmission technology needs to shield the application and transport layers from the complex structure of the network layer and the multi-system physical links below the network layer. The specific technical implementation is as follows:

[0055] The driver design is at the Linux kernel IP layer. The sending and receiving terminals need to first register the virtual network card information, and then register the message sending and receiving functions in NF_IP_HOOK, such as the NF_INET_PRE_ROUTING and NF_INET_POST_ROUTING hook functions, which can complete the parallel transmission, reception, sorting and submission of network packets.

[0056] Example 3

[0057] In this embodiment, the identification information also includes: user sequence number and link sequence number.

[0058] Figure 3 The diagram illustrates the identification information, indicating the content of the encapsulated frame. Here, `frag_flag` represents the segmentation flag (2 bits); `frag_sn` represents the segment sequence number (6 bits); `user_id` represents the user sequence number (4 bits); `link_id` represents the link sequence number (4 bits); `len` represents the original message length (16 bits); `pkt_sn` represents the total message sequence number (32 bits); and `portocol_info` represents the original protocol information, a union with three possible cases: TCP, UDP, and ICMP, each occupying 32 bits.

[0059] Example 4

[0060] In this embodiment, S2, obtaining the target network packet corresponding to each original network packet, includes: determining whether the padded original network packet exceeds the system's maximum transmission unit; if it does not exceed, then the padded original network packet is used as its corresponding target network packet; if it exceeds, then the padded original network packet is cut into multiple segments, and each segment is used as its corresponding target network packet.

[0061] Specifically, when the message sending function intercepts a network packet, it first determines whether the packet needs to be transmitted in parallel. If not, it exits the function without processing. Otherwise, it checks whether the packet, after adding the tail identifier information, exceeds the system's transmission MTU. If it exceeds the MTU, it needs to be segmented into several packets that meet the requirements. If it does not exceed the MTU, it proceeds to the next step. After packet segmentation, each segmented packet is appended with identifier information (total sequence number, link number, segmentation sub-sequence number, segmentation flag, etc.). The tail identifier information is as follows: Figure 2 As shown. The selection of the transmission link is determined by the attribute information of each physical link obtained from the link probe message. The network message will be re-filled with the protocol information of the network message, such as IP address, port information, etc., based on the physical link information. The format of the link probe message is as follows: Figure 3 As shown. After the message information is filled in, the message can be retransmitted using the protocol message sending function.

[0062] Example 5

[0063] In this embodiment, S4 includes: when multiple target network packets are received using the receive packet function, sorting them according to the identification information of each target network packet; determining whether each target network packet is a segmented packet based on the packet segmentation information in the sorted target network packets; if so, reassembling and deleting its identification information according to the packet segmentation information to obtain the corresponding original network packet; otherwise, directly obtaining the corresponding original network packet from its identification information; and finally submitting it to the upper layer to enable it to obtain the service flow.

[0064] Specifically, when the message receiving function intercepts a network message, it first determines whether the message arrived via parallel transmission. If not, it exits the function directly; otherwise, it stores the message in a sorting queue. The sorting thread sorts the message based on its tail identifier and submits it to the processing thread. The processing thread determines whether the current message is a segmented message. If so, it reassembles the message into its original form based on the segment sequence number; otherwise, it removes the tail identifier and submits it to the upper-layer protocol stack, thus completing the reception, processing, and submission of the network message.

[0065] Example 6

[0066] In this embodiment, S3 includes: when sending multiple target network packets corresponding to the service flow, S31: creating multiple ICMP link probe packets and sending them to the receiving side in parallel through multiple links, and obtaining the link attribute information of each parallel link according to the feedback packets from the receiving side; S32: selecting multiple matching links according to the link attribute information of each link, so as to send each target network packet in parallel using the message sending function.

[0067] The IP protocol was not designed for reliable data delivery. Therefore, to ensure the throughput of TCP packets and the transmission quality of UDP packets, a reasonable and effective parallel transmission scheme must be designed to guarantee transmission reliability. In real-time communication, different scenarios have different reliability requirements. This invention summarizes these requirements into the following three categories:

[0068] (1) Best-effort transmission: The system requires the sending and receiving ends to send, receive, and process as many network packets as possible per unit time, and network packets are allowed to be missing during transmission. For example: VR / AR service streams, 4K / 8K high-definition video streams, etc.

[0069] (2) Unordered reliability: The receiving side of the system requires that the data packets from the sending end arrive completely, but there is no requirement for the order in which they arrive. Examples include: file transfer, real-time graphics drawing data, whiteboard writing, and log appending data.

[0070] (3) Orderly and reliable: The system receiver requires that the data packets sent by the sender must arrive in order and completely.

[0071] Since the system needs to carry various types of services and needs to understand the attribute information of the physical link, such as latency, throughput, and packet loss rate, a link probe message is designed in this invention.

[0072] Physical attribute detection is achieved by sending ICMP request messages in the sending-side driver. After the ICMP message is created, the sending side first fills in information such as the packet loss rate, the number of bytes received per unit time, the number of bytes sent per unit time, the control information of the sending side to the user terminal, and the message sending timestamp for a specific link for a certain user terminal. Then, it is sent to the user terminal through this link. After the user terminal recognizes the link detection message, it retains the information of the aggregation server locally and fills in the local number of bytes received and sent per unit time. Then, it returns the message using the automatic return mechanism of ICMP messages. When the aggregation server receives the return message, it can calculate the latency information of the current link using the sending timestamp and the local timestamp, and fill in the information of the sending side locally.

[0073] In summary, ICMP link probe messages can be used to facilitate the exchange of data between user terminals and aggregation server terminals, as well as to enable the server terminal to send and receive control commands from user terminals.

[0074] Example 7

[0075] In this embodiment, S31 includes: creating an ICMP link probe packet for each link on the sending side; the ICMP link probe packet carries information including: packet loss rate, number of bytes received per unit time, number of bytes sent per unit time, control information from the sending side to the receiving side, and message sending timestamp information for a certain link; sending each ICMP link probe packet to the receiving side through the corresponding link, so that the receiving side parses the ICMP link probe packet of each link and retains the information of the sending side for local use, fills in the local confirmation of the number of bytes received per unit time, the number of bytes sent per unit time, and the control information from the sending side to the sending side, and returns it using the automatic return mechanism of ICMP messages; after receiving the return message, the sending side calculates the link attribute information of each link using the sending timestamp and the local timestamp, and retains the information filled in by the receiving side for local use.

[0076] Example 8

[0077] In this embodiment, the information carried by the ICMP link probe packet includes: user ID number, link ID number, link transmission valid flag bit, control command valid flag bit, reserved bit, link probe message flag bit, packet classification, link delay of the last probe, total amount of local link transmission in the last probe cycle, total amount of local link reception in the last probe cycle, pointer structure for recording time, and local absolute time of sending probe request packet.

[0078] Figure 4 This is a schematic diagram of the ICMP link probe packet format, containing various link information needed by the system, such as flags, link latency, and bytes sent and received per unit time. Specifically: `user_id` represents the user ID (3 bits); `link_id` represents the link ID (2 bits); `is_used` is a link transmission validity flag, indicating whether the link in this probe packet can transmit valid data (1 bit); `is_ctrl` is a control command validity flag, indicating whether the control command received by the user has been completed (1 bit). `rsv` is a reserved bit (1 bit); the `type` field identifies whether the packet is a link probe packet (2 bytes); the `code` field is the packet category (2 bytes); the `udelay` field is the link latency of the last probe (4 bytes); `snd_bw` is the total amount of data sent by the local link in the previous probe cycle (4 bytes); `rcv_bw` is the total amount of data received by the local link in the previous probe cycle (4 bytes). The `*ts` field is a pointer structure recording the local absolute time of sending the probe request packet (8 bytes).

[0079] Example 9

[0080] In this embodiment, S32 includes: the sending side selects the link that matches each target network packet from all the link attribute information based on the link attribute information of all the links, fills the protocol information according to the link attribute information of the link that matches each target network packet, and completes the sending of the packet using the packet sending function through the protocol information; the protocol information includes IP address and port information.

[0081] Example 10

[0082] This embodiment provides a multi-link parallel network communication device, including:

[0083] Create a module to create send and receive message functions at IP layer hook points;

[0084] The acquisition module is used to fill in the identification information in each original network packet corresponding to each service flow in order to obtain the target network packet corresponding to each original network packet. The identification information includes: total sequence number of the packet, original length of the packet, original protocol information of the packet and packet segmentation information.

[0085] The acquisition module is used to fill in the identification information in each original network packet corresponding to each service flow in order to obtain the target network packet corresponding to each original network packet. The identification information includes: total sequence number of the packet, original length of the packet, original protocol information of the packet and packet segmentation information.

[0086] The sending module is used to select multiple links and use the sending message function to transmit each target network message in parallel when sending multiple target network messages corresponding to the service flow.

[0087] The receiving module is used to restore the original network packets in sequence according to the identification information in each target network packet when multiple target network packets are received using the receiving message function, and submit them to the upper layer so that they can obtain the service flow.

[0088] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-link parallel network communication method, characterized in that, include: S1: Create send and receive message functions at the IP layer hook point; S2: Fill identification information into each original network packet corresponding to each service flow to obtain the target network packet corresponding to each original network packet; The identification information includes: total message sequence number, original message length, original message protocol information, and message segmentation information; S3: When sending multiple target network packets corresponding to the service flow, select multiple links and use the packet sending function to transmit each target network packet in parallel; S4: When multiple target network packets are received using the received packet function, each target network packet is restored into the original network packets arranged in sequence according to the identification information in each target network packet, and submitted to the upper layer so that it can obtain the service flow; S2 includes: filling identification information at the end of each original network packet corresponding to the service flow to obtain the target network packet corresponding to each original network packet; S2, obtaining the target network packet corresponding to each of the original network packets, includes: determining whether the padded original network packet exceeds the system's maximum transmission unit; if it does not exceed, then using the padded original network packet as its corresponding target network packet; if it exceeds, then cutting the padded original network packet into multiple segments, and using each segment as its corresponding target network packet. S4 includes: when multiple target network packets are received using the received packet function, sorting them according to the identification information of each target network packet; determining whether each target network packet is a segmented packet according to the packet segmentation information in the sorted target network packets; if so, reassembling and deleting its identification information according to the packet segmentation information to obtain the corresponding original network packet; otherwise, directly obtaining the corresponding original network packet according to its identification information; and finally submitting it to the upper layer to obtain the service flow.

2. The multi-link parallel network communication method as described in claim 1, characterized in that, The identification information also includes: user sequence number and link sequence number.

3. The multi-link parallel network communication method as described in claim 1, characterized in that, S3 includes: when sending multiple target network packets corresponding to the service flow. S31: Create multiple ICMP link probe packets and send them to the receiving side in parallel through multiple links, and obtain the link attribute information of each parallel link based on the feedback message from the receiving side; S32: Select multiple matching links based on the link attribute information of each link, and use the message sending function to send each of the target network messages in parallel.

4. The multi-link parallel network communication method as described in claim 3, characterized in that, S31 includes: On the sending side, an ICMP link probe packet is created for each link. The ICMP link probe packet carries information including: the packet loss rate for a certain link, the number of bytes received per unit time, the number of bytes sent per unit time, the control information from the sending side to the receiving side, and the message sending timestamp information. Each ICMP link probe packet is sent to the receiving side through the corresponding link, so that the receiving side parses the ICMP link probe packet for each link, retains the information from the sending side for local use, fills in the local confirmation of the number of bytes received per unit time, the number of bytes sent per unit time, and the control information from the sending side to the sending side, and returns it using the automatic ICMP message return mechanism. After receiving the return message, the sending side calculates the link attribute information of each link using the sending timestamp and the local timestamp, and retains the information filled in by the receiving side for local use.

5. The multi-link parallel network communication method as described in claim 4, characterized in that, The ICMP link probe packet carries the following information: user ID number, link ID number, link transmission valid flag bit, control command valid flag bit, reserved bit, link probe message flag bit, packet classification, link delay of the last probe, total amount of local link transmission in the last probe cycle, total amount of local link reception in the last probe cycle, pointer structure for recording time, and local absolute time of sending probe request packet.

6. The multi-link parallel network communication method as described in claim 3, characterized in that, S32 includes: The sending side selects the link that matches each target network packet based on the link attribute information of all links, fills the protocol information according to the link attribute information of the matching link for each target network packet, and completes the packet transmission using the packet transmission function through the protocol information; the protocol information includes IP address and port information.

7. A multi-link parallel network communication device, characterized in that, A method for performing the multi-link parallel network communication method according to any one of claims 1-6, comprising: Create a module to create send and receive message functions at IP layer hook points; The acquisition module is used to fill identification information into each original network packet corresponding to each service flow in order to obtain the target network packet corresponding to each original network packet; the identification information includes: total sequence number of the packet, original length of the packet, original protocol information of the packet and packet segmentation information. The sending module is configured to select multiple links and use the sending message function to transmit each target network message in parallel when sending multiple target network messages corresponding to the service flow; The receiving module is configured to, when receiving multiple target network packets using the receiving message function, restore each target network packet into the original network packets arranged in sequence according to the identification information in each target network packet, and submit them to the upper layer so that it can obtain the service flow.

Images