Data packet deduplication method, apparatus, device, storage medium and program product

Abstract

The present application belongs to the technical field of networks. Disclosed are a data packet deduplication method, an apparatus, a device, a storage medium and a program product. The method comprises: receiving a first data packet; and, if the first data packet carries a retransmission identifier, deduplicating the first data packet on the basis of a deduplication matching table, such that there is one first data packet in a receiving end device, the deduplication matching table storing information about data packets that have been received by the receiving end device. It can be seen that the receiving end device in the present application only deduplicates a small number of data packets carrying retransmission identifiers, not only ensuring that the receiving end device can process exactly once the data packets forwarded by a network, but also realizing lightweight deduplication of the receiving end device.

Description

Data packet deduplication method, device, equipment, storage medium and program product

[0001] The present application claims priority to the Chinese patent application No. 202410866893.1, filed on June 29, 2024, and entitled "Data packet deduplication method, device, equipment, storage medium and program product", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of network technology, in particular to a data packet deduplication method, device, equipment, storage medium and program product. BACKGROUND

[0003] In a unified bus (UB) protocol, a data link layer supports sending and receiving data messages and link layer control messages, and a minimum transmission unit (Flit) of the messages is fixed as 20 bytes. When a first forwarding device sends a data packet to a receiving device, the first forwarding device backs up a Flit of the data packet to a local retry buffer (also referred to as Retry-Buff) after sending the Flit from a corresponding port, and a second forwarding device at a next hop receives the Flit and performs cyclic redundancy check (CRC) or decoding on the Flit. When the CRC is correct or the forward error correction (FEC) decoding is successful, the second forwarding device returns an acknowledgement (ACK) message to the first forwarding device to inform the first forwarding device to release the retry buffer and continue forwarding the Flit to the receiving device. When the first forwarding device receives the ACK message, the first forwarding device deletes the Flit from the retry buffer. However, when the second forwarding device fails or a communication path between the first forwarding device and the second forwarding device fails, the first forwarding device can not receive the ACK message fed back by the second forwarding device, and the first forwarding device determines a backup communication path through a fast reroute (FRR) mechanism and re-sends the Flits in the retry buffer to the receiving device along the backup communication path. The receiving device can receive duplicate data packets.

[0004] Therefore, there is an urgent need for a data packet deduplication method to ensure that the receiving device can process the network forwarded data packet exactly once. SUMMARY

[0005] The application provides a data packet deduplication method, device, equipment, storage medium and program product, so that the receiving end equipment performs deduplication processing on the data packet carrying the retransmission identifier after receiving the data packet, which can not only ensure that the receiving end equipment can process the network forwarded data packet exactly once, but also realizes the lightweight deduplication of the receiving end equipment on the data packet. The technical solution is as follows:

[0006] In a first aspect, a data packet deduplication method is provided, and the method comprises:

[0007] receiving a first data packet; if the first data packet carries a retransmission identifier, performing deduplication processing on the first data packet according to a deduplication matching table, so that there is only one first data packet in the receiving end equipment, and the deduplication matching table stores information of the data packet received by the receiving end equipment.

[0008] In a possible implementation, if the first data packet does not carry a retransmission identifier, the first data packet is retained. Further, in the case of retaining the first data packet, the deduplication matching table is updated according to the information of the first data packet, that is, the information of the first data packet is recorded in the deduplication matching table.

[0009] It should be noted that for the receiving end equipment, the deduplication matching table maintained and managed by the receiving end equipment can only store information of the data packet received in a specified time period, or can continuously store information of all received data packets, and the application does not limit the time range of the information of the received data packet stored in the deduplication matching table, nor the amount of information in the deduplication matching table.

[0010] As can be seen, for the receiving end equipment, if the data packet received by the receiving end equipment carries a retransmission identifier, it means that the data packet is a retransmitted data packet in the network, and for the retransmitted data packet, the receiving end equipment may have received the same data packet before. Therefore, in the case that the data packet carries a retransmission identifier, the receiving end equipment needs to perform deduplication processing on the data packet. In this way, it can be ensured that there is no duplicate data packet in the receiving end equipment, so that the receiving end equipment can process the network forwarded data packet exactly once.

[0011] Moreover, compared with a scheme in the related art that detects a repeated packet based on a packet sequence number (PSN) and judges whether each data packet is a repeated data packet according to the PSN carried by the data packet, the receiving end device in the application can directly judge whether the data packet is a retransmission data packet according to whether the data packet carries a retransmission identifier in the process of receiving the data packet, and then only needs to perform deduplication processing on a small number of data packets marked with the retransmission identifier based on the deduplication matching table, without performing the operation of judging whether each data packet is a repeated data packet, thereby greatly reducing the resource consumption of the receiving end device and realizing lightweight deduplication of the receiving end device on the data packet.

[0012] In a possible implementation, the deduplication processing of the first data packet according to the deduplication matching table includes: if it is determined that the first data packet is a repeated data packet according to the deduplication matching table, discarding the first data packet; and if it is determined that the first data packet is a non-repeated data packet according to the deduplication matching table, retaining the first data packet.

[0013] That is, in the case where the first data packet carries the retransmission identifier, it is necessary to further confirm whether the receiving end device has received the first data packet based on the deduplication matching table, to determine whether the first data packet is a repeated data packet based on the deduplication matching table.

[0014] In a possible implementation, the retaining of the first data packet includes: storing the first data packet, and recording information of the first data packet in the deduplication matching table to update the deduplication matching table.

[0015] That is, in the case where the first data packet is not a repeated data packet, it is necessary to update the deduplication matching table based on the first data packet, so as to avoid repeated reception of the first data packet in the process of subsequently receiving data packets.

[0016] In a possible implementation, the method further includes: receiving a first message carrying information of a forwarded data packet, the forwarded data packet being a data packet forwarded by a target forwarding device to the receiving end device in a target fault detection period, the first message being used to indicate that a target communication path between a sending end device and the receiving end device is detected to have a fault in the target fault detection period; and updating the deduplication matching table based on the information of the forwarded data packet.

[0017] The target forwarding device stores information of all the data packets forwarded in the target fault detection period, and can detect whether the communication path of the data packets is faulty during forwarding of the data packets. Based on this, if the target communication path is detected to be faulty in the target fault detection period, a first message can be constructed and sent to the receiving end device to prompt the receiving end device that there are retransmitted data packets in the data packets received subsequently, which need to be processed for deduplication.

[0018] Moreover, the first message carries information of the data packets forwarded in the target fault detection period in which the target communication path is detected to be faulty, so that the receiving end device can update the deduplication matching table based on the information of the data packets, and determine whether there are repeated data packets retransmitted in the target fault detection period in the data packets received subsequently based on the updated deduplication matching table.

[0019] In a possible implementation, the first message includes a first field carrying a message type, and the message type indicates that the first message is a deduplication trigger message used to instruct the receiving end device to perform deduplication processing on the data packets received after the first message.

[0020] That is, the first message can be a deduplication trigger message. After the receiving end device receives the first message, the receiving end device can not only parse the information of the data packets from the first message, but also perform deduplication processing on the data packets received after the first message based on the message type of the first message.

[0021] In other words, the first message as a deduplication trigger message can instruct the receiving end device to start the deduplication processing operation.

[0022] In a possible implementation, the target forwarding device is located on the target communication path, and includes at least one forwarding device detecting that a target device or a target path segment on the target communication path is faulty. If the target device is faulty, the target forwarding device includes at least one forwarding device connected with the target device. If the target path segment is faulty, the target forwarding device includes a first forwarding device and / or a second forwarding device, the second forwarding device is a next-hop forwarding device on the target communication path connected with the first forwarding device, and the target path segment is a path segment between the first forwarding device and the second forwarding device.

[0023] That is, the fault in the target communication path includes a fault of a link corresponding to a path segment or a fault of a forwarding device on the target communication path. When the link corresponding to the path segment has the fault, two forwarding devices connected by the link can detect the fault of the link, and the two forwarding devices detecting the fault of the link can construct and send the first message. When the forwarding device on the target communication path has the fault, other forwarding devices connected to the forwarding device can detect the fault of the forwarding device, and one or more other forwarding devices detecting the fault of the forwarding device can construct and send the first message.

[0024] In a possible implementation, the information of the forwarded data packet includes identification information of the forwarded data packet, and the identification information includes a transaction layer segment number of the forwarded data packet or packet sequence number information of the forwarded data packet.

[0025] It should be noted that the present application only uses the transaction layer segment number of the data packet or the packet sequence number information to uniquely identify a data packet. In actual application, other information can also be used to identify and distinguish data packets for better distinguishing data packets. The present application does not limit the specific content of the identification information of the data packet.

[0026] In a possible implementation, the de-duplication trigger message includes a first field, and the first field is used to carry the identification information of the forwarded data packet.

[0027] In a possible implementation, the information of the forwarded data packet further includes device address information, and the device address information at least indicates an address of the sending-end device.

[0028] In a possible implementation, the de-duplication trigger message includes a first field and a second field, the first field is used to carry the identification information of the forwarded data packet, and the second field is used to carry the device address information.

[0029] In a possible implementation, the first data packet further carries an end-retransmission identifier, and the end-retransmission identifier indicates that the first data packet is the last data packet for de-duplication processing.

[0030] That is, the first data packet can carry the retransmission identifier and the end-retransmission identifier at the same time, to indicate that the first data packet is the last retransmitted data packet and the last data packet for de-duplication processing. For the receiving-end device, based on the end-retransmission identifier carried by the first data packet, after de-duplication processing of the first data packet, no de-duplication processing is performed on the subsequently received data packet.

[0031] In other words, in this application, the end retransmission identifier can be used to indicate the receiving end device to end the deduplication operation.

[0032] In a second aspect, a data packet deduplication method is provided, which is applied to a target forwarding device, and the method comprises:

[0033] The first data packet is sent to the receiving end device to instruct the receiving end device to perform deduplication processing on the first data packet according to a deduplication matching table, so that there is one first data packet in the receiving end device; wherein the first data packet carries a retransmission identifier, and the deduplication matching table stores information of data packets that have been received by the receiving end device.

[0034] As can be seen, for a transmitted data packet, if the data packet is a data packet retransmitted by the forwarding device, the data packet carries a retransmission identifier. In other words, for the forwarding device in the network, when forwarding data packets to the receiving end device, a small number of retransmitted data packets are marked to carry a retransmission identifier.

[0035] Moreover, compared with the scheme in the related art that detects duplicate packets based on PSN, and needs to judge whether each data packet is a duplicate data packet according to the PSN carried by the data packet, since the forwarding device in this application can mark a small number of retransmitted data packets, the receiving end device can directly judge whether the data packet is a retransmitted data packet according to whether the data packet carries a retransmission identifier, and then only needs to perform deduplication processing on a small number of data packets marked with a retransmission identifier based on the deduplication matching table, without performing the operation of judging whether each data packet is a duplicate data packet, which greatly reduces the resource consumption of the receiving end device, and realizes the lightweight deduplication of the receiving end device on data packets.

[0036] In a possible implementation, the method further comprises:

[0037] The first message is sent to the receiving end device to instruct the receiving end device to update the deduplication matching table based on information of the data packets carried by the first message; wherein the data packets are data packets retransmitted by the target forwarding device to the receiving end device within a target fault detection period, the first message is used to indicate that a target communication path between the sending end device and the receiving end device is detected to have a fault within the target fault detection period.

[0038] The target forwarding device stores information of all the forwarded data packets in the target failure detection period, and can detect whether the communication path of the transmitted data packet is faulty during forwarding of the data packet. Based on this, if the target communication path is detected to be faulty in the target failure detection period, a first message can be constructed and sent to the receiving end device to prompt the receiving end device that there are retransmitted data packets in the data packets received subsequently, which need to be processed for deduplication.

[0039] In a possible implementation, the first message includes a first field, and the first field carries a message type, which indicates that the first message is a deduplication trigger message, and the deduplication trigger message is used to instruct the receiving end device to perform deduplication processing on the data packets received after the first message.

[0040] In a possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects that a target device or a target path segment on the target communication path is faulty; if the target device is faulty, the target forwarding device includes at least one forwarding device connected with the target device; if the target path segment is faulty, the target forwarding device includes a first forwarding device and / or a second forwarding device, the second forwarding device is a next-hop forwarding device on the target communication path connected with the first forwarding device, and the target path segment is a path segment between the first forwarding device and the second forwarding device.

[0041] In a possible implementation, the information of the forwarded data packet includes identification information of the forwarded data packet, and the identification information includes a transaction layer segment number of the forwarded data packet or packet sequence number information of the forwarded data packet.

[0042] In a possible implementation, the information of the forwarded data packet further includes device address information, and the device address information at least indicates an address of the sending end device.

[0043] In a possible implementation, the first data packet further carries an end retransmission identifier, and the end retransmission identifier indicates that the first data packet is the last data packet for deduplication processing.

[0044] In a third aspect, a data packet deduplication apparatus is provided, which has functions to implement behaviors of the data packet deduplication method in the first aspect. The data packet deduplication apparatus includes at least one module for implementing steps of the data packet deduplication method provided in the first aspect.

[0045] In a fourth aspect, a data packet deduplication apparatus is provided, which has functions to implement the behaviors of the data packet deduplication method in the second aspect. The data packet deduplication apparatus comprises at least one module for implementing the steps of the data packet deduplication method provided in the second aspect.

[0046] In a fifth aspect, a computer device is provided, which comprises a processor and a memory. The memory is configured to store a computer program for executing the data packet deduplication method provided in the first aspect, or a computer program for executing the data packet deduplication method provided in the second aspect. The processor is configured to execute the computer program stored in the memory, so as to implement the data packet deduplication method in the first aspect, or implement the data packet deduplication method in the second aspect.

[0047] In a possible implementation, the computer device can further comprise a communication bus for establishing a connection between the processor and the memory.

[0048] In a sixth aspect, a computer readable storage medium is provided, which stores instructions. When the instructions are executed on a computer, the computer is caused to perform the steps of the data packet deduplication method in the first aspect, or perform the steps of the data packet deduplication method in the second aspect.

[0049] In a seventh aspect, a computer program product is provided, which comprises instructions. When the instructions are executed on a computer, the computer is caused to perform the steps of the data packet deduplication method in the first aspect, or perform the steps of the data packet deduplication method in the second aspect.

[0050] Alternatively, a computer program is provided. When the computer program is executed on a computer, the computer is caused to perform the steps of the data packet deduplication method in the first aspect, or perform the steps of the data packet deduplication method in the second aspect.

[0051] The technical effects obtained by the second aspect to the seventh aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and thus are not described herein. BRIEF DESCRIPTION OF DRAWINGS

[0052] FIG. 1 is a flow diagram of a process of transmitting and receiving data packets at a link layer according to an embodiment of the present application;

[0053] FIG. 2 is a diagram of data packet forwarding in a case of link failure in a first network topology according to an embodiment of the present application;

[0054] FIG. 3 is a diagram of data packet forwarding in a case of link failure in a second network topology according to an embodiment of the present application;

[0055] FIG. 4 is a schematic diagram of a layer hierarchy of a UB-G protocol stack according to an embodiment of the present application;

[0056] FIG. 5 is a schematic diagram of a flow of detecting duplicate packets using a packet sequence number according to a related art;

[0057] FIG. 6 is a schematic diagram of a layer hierarchy of a UB-C protocol stack according to an embodiment of the present application;

[0058] FIG. 7 is a schematic diagram of an architecture of a data center according to an embodiment of the present application;

[0059] FIG. 8 is a schematic diagram of an application scenario of a data packet deduplication method according to an embodiment of the present application;

[0060] FIG. 9 is a schematic diagram of a structure of a network device according to an embodiment of the present application;

[0061] FIG. 10 is a schematic diagram of a structure of a computer device according to an embodiment of the present application;

[0062] FIG. 11 is a flowchart of a data packet deduplication method according to an embodiment of the present application;

[0063] FIG. 12 is a schematic diagram of retransmitting a data packet through a backup path according to an embodiment of the present application;

[0064] FIG. 13 is a schematic diagram of a format of a first message according to an embodiment of the present application;

[0065] FIG. 14 is a schematic diagram of a transmission path of a first message based on a first network topology according to an embodiment of the present application;

[0066] FIG. 15 is a schematic diagram of a transmission path of another first message based on the first network topology according to an embodiment of the present application;

[0067] FIG. 16 is a schematic diagram of a transmission path of a first message based on a second network topology according to an embodiment of the present application;

[0068] FIG. 17 is a schematic diagram of a transmission path of another first message based on the second network topology according to an embodiment of the present application;

[0069] FIG. 18 is a schematic diagram of a format of another first message according to an embodiment of the present application;

[0070] FIG. 19 is a schematic diagram of a format of yet another first message according to an embodiment of the present application;

[0071] FIG. 20 is a schematic diagram of a structure of a data packet header according to an embodiment of the present application;

[0072] FIG. 21 is a logic diagram illustrating a process of a receiving end device processing a data packet according to an embodiment of the present application;

[0073] FIG. 22 is a flowchart illustrating a process of a switch processing a data packet according to an embodiment of the present application;

[0074] FIG. 23 is a logic diagram illustrating a process of releasing an interval of a retransmission buffer according to an embodiment of the present application;

[0075] FIG. 24 is a flowchart illustrating a process of a receiving end device responding to a deduplication trigger message according to an embodiment of the present application;

[0076] FIG. 25 is a flowchart illustrating a process of a receiving end device deduplicating a data packet according to an embodiment of the present application;

[0077] FIG. 26 is a structural diagram illustrating a data packet deduplication apparatus according to an embodiment of the present application;

[0078] FIG. 27 is a structural diagram illustrating another data packet deduplication apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0079] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will further describe the embodiments of the present application in conjunction with the accompanying drawings.

[0080] To facilitate understanding, before explaining the data packet deduplication method provided by the embodiments of the present application, the application background and implementation environment involved in the embodiments of the present application are introduced.

[0081] First, the related background of the embodiments of the present application is introduced.

[0082] Referring to FIG. 1, in a unified bus (UB) protocol, a data link layer supports sending and receiving service messages and link layer control messages. The minimum unit of the data link layer transmission message is a Flit, and the size of 1 Flit is fixed at 20 bytes. As shown in FIG. 1, after a first forwarding device transmits a certain Flit of a data packet from a certain port, the Flit that has been transmitted is backed up to a local retransmission buffer (also referred to as Retry-Buff). A second forwarding device at a next hop in the network performs a cyclic redundancy check (CRC) check or decoding on the Flit after receiving the Flit. When the CRC check is correct or forward error correction (FEC) decoding is successful, an acknowledgement (ACK) message is returned to the first forwarding device to inform the first forwarding device to release the retransmission buffer and continue to forward the Flit to a receiving end device. Further, after receiving the ACK message sent by the second forwarding device, the first forwarding device deletes the backup of the Flit from the retransmission buffer to release the space resource of the retransmission buffer.

[0083] wherein the CRC check is a common error detection method used to check whether an error has occurred in the data transmission process. If the CRC check is correct, it means that the Flit has not occurred an error in the transmission process. The FEC decoding is a more advanced error correction method, which can not only detect errors, but also correct these errors in some cases. If the FEC decoding is successful, it means that the Flit has been successfully corrected even if there is an error.

[0084] In a possible implementation, the first forwarding device and the second forwarding device can be switches, routers, or other network devices capable of forwarding data packets, such as a switching module on a computing chip, and the embodiments of the present application do not limit this.

[0085] In a possible implementation, the ACK message can be carried in the header of a link layer packet, such as an ACK field segment of a link packet header (LPH) or a link block header (LBH); or the ACK message can also be sent by a special Flit, such as a CRD (full name: credit, a flow control mechanism) message, to inform the first forwarding device that the data packet Flit has been successfully received.

[0086] It should be understood that the LPH, as the header of the link layer packet, contains various information about the packet, such as the source address, the destination address, the data length, the protocol type, and the like. In the data transmission process, the LPH can help the network device to identify and process the packet. When a large packet (for example, a length exceeding 32 Flits) is divided into multiple data blocks for segmented transmission, the LBH refers to the header of a data block, and similar to the LPH, the LBH also contains information about the data block, which can include the length, type, and checksum of the data block. In the communication protocol, the CRD is usually used for flow control, indicating the amount of data that the receiving end can receive. In simple terms, the CRD is a counter or token system used to prevent the sending end from sending too much data and causing the receiving end buffer to overflow, so when the receiving end has enough buffer to receive new data, it will send a CRD message to the sending end to tell the sending end how much data it can continue to send.

[0087] In the process of transmitting the data packet Flit in the data link layer, if the CRC check of the Flit fails or the FEC decoding fails, the second forwarding device will request the first forwarding device to retransmit the Flit. In response to the retransmission request, the first forwarding device retransmits the backup data packet Flit in the retransmission buffer to the second forwarding device.

[0088] Similarly, for a retransmission-based data packet Flit, if the CRC check is correct or the forward error correction (FEC) decoding is successful, the second forwarding device returns an ACK message to the first forwarding device to inform the first forwarding device to release the retransmission buffer and continue forwarding the Flit to the receiving end device.

[0089] In practical applications, bus protocols are more commonly used within a machine frame, such as a peripheral component interconnect express (PCI-E). In the machine frame, devices are connected by cables, and the communication reliability is very high. However, as the size of the bus network increases, optical modules need to be used to realize cross-cabinet connection. However, the native reliability of the optical port is more than one order of magnitude lower than that of the cable. Therefore, if a large number of optical ports are used in a network system, the flash interruption (i.e., optical signal interruption) problem may occur.

[0090] As an example, in a network containing 12 blocks, each block contains about 32 servers, and each server has 40 ports (which may be an abstraction of network interface cards or CPU cores, etc.) upstream. In this case, the entire network contains about 12*32*40 = 15360 servers. For this network, the probability of port state change (such as port up and down, which may be caused by network failure, server reboot, etc.) per hour is 3 times, that is, on average, three ports will change state within an hour, whether due to optical flash interruption or server power on / off. In addition, the probability of flash interruption of an optical module is 2.1%, which is usually caused by cable breakage, optical module failure, dust pollution, etc. Based on this, in a 1024P (with 1024 processing units or ports) supernode, the number of optical modules can reach 8000, which means that in a large supernode, a large number of optical modules are used for data communication, and according to the above optical module flash interruption probability, the expected number of optical module flash interruptions per day in a 1024P supernode is 8000*2.1% / 365 = 0.46 times. As can be seen, in a network system using optical ports, a certain number of flash interruption events may occur every day.

[0091] As an example, referring to FIG. 2, in the first network topology (2D Torus networking architecture), node 8 is communicating with node 7 via the primary path, and after the link between node 9 and node 10 fails, node 10 sends all the accumulated data packets to node 7. After detecting the link failure, node 9 determines a backup path through fast reroute (FRR) and resends the data packets in the retransmission buffer to node 7 along the backup path.

[0092] When data packets are transmitted between node 8 and node 7, the main path of the communication can be node 8-node 9-node 10-node 11-node 7, and the backup path of the communication can be node 8-node 9-node 5-node 6-node 7.

[0093] However, when node 9 retransmits the data packets in the retransmission buffer, the data packets can include data packets that have passed through node 10 and are sent to node 7, but node 10 fails to return an ACK message due to a link failure. At this time, node 7 can receive duplicate data packets, which can destroy data consistency, atomicity, concurrency control, and cause resource waste, and can affect the correctness of write and atomic operations, and cannot guarantee that node 7 can process the network-transmitted data packets exactly once.

[0094] As another example, referring to FIG. 3, in a second network topology (Clos network architecture), node A in the processor unit is communicating with node H via a main path, and after a link failure between node Leaf 1 in the access layer and node Spine 1 in the core layer in the network topology, node Spine 1 sends all the data packets to node H. After detecting the link failure, node Leaf 1 determines a backup path through FRR and retransmits the data packets in the retransmission buffer to node H along the backup path.

[0095] When data packets are transmitted between node A and node H, the main path of the communication can be node A-node Leaf 1-node Spine 1-node Leaf n-node H, and the backup path of the communication can be node A-node Leaf 1-node Spine 16-node Leaf n-node H.

[0096] However, when node Leaf 1 retransmits the data packets in the retransmission buffer, the data packets can include data packets that have passed through node Spine 1 and are sent to node H, but node Spine 1 fails to return an ACK message due to a link failure. At this time, node H can receive duplicate data packets, which can destroy data consistency, atomicity, concurrency control, and cause resource waste, and can affect the correctness of write and atomic operations, and cannot guarantee that node H can process the network-transmitted data packets exactly once.

[0097] In summary, when multiple nodes in a UB system communicate through a network, how to guarantee that the receiving end device can process the network-transmitted data packets exactly once becomes a technical problem to be solved.

[0098] To solve the above technical problems, the related art also provides a corresponding solution. To facilitate the comparison of the solution provided by the embodiments of the present application, the solution provided by the related art is first introduced briefly.

[0099] In a related art, please refer to the UB-G domain (UB-Global, referred to as UB-G) protocol stack diagram shown in FIG. 4. The complete transport layer in UB-G is used to provide reliable transmission service and unreliable transmission service. The UB-G domain refers to a specific functional domain or system range based on UB-G technology or standard. In a distributed system, a domain usually refers to a collection of system components or services that are logically related or have common functional characteristics.

[0100] Among them, the UB reliable transmission service detects duplicate packets and network packet loss through the packet sequence number (Packet Sequence Number, PSN) to achieve the purpose of reliable transmission of data packets. Among them, the PSN is carried in the transport packet header (transport packet header, TPH). The receiving end device returns an ACK message (also referred to as TP ACK message) for each transport (transport, referred to as TP) connection. The value of PSN is incremented by 1. In a specific implementation, the sending end device and the receiving end device negotiate the initial value of PSN when establishing a TP connection. The initial value of PSN is randomly between 0 and 16M-1. Among them, the initial value of PSN is the PSN value carried by the first data packet.

[0101] In an implementation mode, when the receiving end device returns the TP ACK message, the TP ACK message can carry the accumulated PSN value to indicate that all the data packets with the accumulated PSN have been correctly received. For example, if the receiving end device receives 10 data packets from the sending end device, the PSN values of the 10 data packets are from 1 to 10. At this time, the receiving end device can aggregate all the ACK messages into one message, that is, a TP ACK message with a PSN value of 10, to inform the sending end device that the data packet with the PSN value of 10 and all the previous data packets have been correctly received by sending the TP ACK message to the sending end device.

[0102] It should be noted that the PSN value only acts within a TP connection. The PSN values of different TP connections are completely independent and have no association.

[0103] As an example, referring to FIG. 5, the bit width of the PSN value is 24 bits, representing a range of 0-16M-1; the maximum number of outstanding requests of the PSN value issued by the sending device is 8M, i.e., the maximum PSN value issued minus the minimum PSN value waiting for a response is no more than 8M-1. Among them, the outstanding requests of the sending device refer to the number of uncompleted requests that can be submitted to the memory or other target device at a given time period, which may include data packets, read / write operations, etc., which have been sent but have not yet received an acknowledgement or completion.

[0104] As shown in FIG. 5, if the minimum PSN value waiting for a response is 0, the maximum PSN value allowed to be sent is 8M-1 (i.e., 8388607); if the minimum PSN value waiting for a response is 8M (i.e., 8388608), the maximum PSN value allowed to be sent is 16M-1 (i.e., 16777215); if the minimum PSN value waiting for a response is 10M (i.e., 1048576), the maximum PSN value allowed to be sent is 18M-1 (i.e., 18874367), since 18M-1 exceeds the representation range of 24 bits, there is an overflow of high bits, i.e., the maximum PSN allowed to be sent is 2M-1 (i.e., 2097151).

[0105] For the receiving device, the expected PSN (EPSN) refers to the expected in-sequence PSN. For example, the data packet with PSN value = 5 has been successfully received by the receiving device, and the data packet with PSN value = 5 is not an out-of-sequence packet, then the EPSN value of the receiving device is 6.

[0106] In addition, the sending device will not send outstanding requests with PSN values exceeding 8M, so the interval [EPSN-8M, EPSN-1] corresponding to the EPSN value is the duplicate packet interval.

[0107] Based on this, the implementation process of the exactly once of the receiving end device can be: after receiving the data packet, the receiving end device first judges whether the data packet falls in the repetition interval, the out-of-order interval or the invalid interval based on the PSN value of the data packet. If the PSN value of the data packet falls in the repetition interval, the data packet is discarded, and an ACK message is returned to the sending end device, wherein the PSN value carried in the ACK message is equal to the EPSN value of the receiving end device minus 1; if the PSN value of the data packet is equal to the EPSN value of the receiving end device, the data packet is received, and an ACK message is returned to the sending end device, wherein the PSN value carried in the ACK message is equal to the EPSN value of the receiving end device; if the PSN value of the data packet falls in the maximum allowed out-of-order interval, the data packet is received, and a selective ACK (SACK) message is returned to the sending end device; if the PSN value of the data packet falls in the invalid interval, the data packet is directly discarded.

[0108] However, in the above solution, all the data packets received by the receiving end device pass through the UB-G transport layer, and the resource area of the UB-G transport layer accounts for 3 square millimeters of the resource area of the receiving end device, and the processing speed is 200Mpps, wherein pps stands for packet per second, which means how many packet data packets are sent per second. It can be seen that, in order to achieve the throughput requirement of 1Gpps in the UB-C domain (UB-Clan, referred to as UB-C), the above solution needs to additionally increase the resource area of 12 square millimeters in the receiving end device, which is too large in resource overhead, resulting in that the receiving end device cannot bear too much load.

[0109] Similarly, UB-C here refers to a specific set or domain based on UB bus technology.

[0110] In another related technology, please refer to the UB-C protocol stack diagram shown in FIG. 6. In the UB-C protocol stack, the lightweight transport layer has no TP connection, and its reliable transmission service relies on the retransmission mechanism of the UB link layer to ensure that there is no packet loss in the end-to-end transmission of the data packet.

[0111] However, the lightweight transport layer in the above UB-C protocol can only solve the problem of packet loss detection, but cannot solve the problem of repeated packets received by the receiving end device, that is, it cannot guarantee that the receiving end device can process the network forwarded data packet exactly once.

[0112] Based on the above two solutions, the embodiment of the present application provides a data packet deduplication method, which can be applied to a lightweight transport layer in a UB-C. In the data packet deduplication method provided by the embodiment of the present application, after detecting that the target communication path of the current transmission data packet has a fault and performing FRR, the forwarding device in the network marks the data packet retransmitted along the backup path next time with a retransmission identifier to indicate that the receiving end device that the data packet is a retransmission data packet. Based on this, after receiving the data packet, the receiving end device parses the data packet to determine whether the retransmission identifier is carried in the data packet. If the data packet carries the retransmission identifier, the receiving end device performs deduplication processing on the data packet according to the deduplication matching table, so that there is no duplicate data packet in the receiving end device. The deduplication matching table stores information of the data packet received by the receiving end device. As can be seen, the embodiment of the present application only performs deduplication processing on a small amount of data packets carrying the retransmission identifier, and compared with the scheme of judging and processing each data packet based on the PSN value in the UB-G, the embodiment of the present application realizes lightweight deduplication and ensures that the receiving end device can process the data packet forwarded by the network exactly once.

[0113] Next, the application scenarios and implementation environments related to the embodiment of the present application are introduced.

[0114] The data packet deduplication method provided by the embodiment of the present application can be applied in a data center scenario. Referring to FIG. 7, the data center can be a UB-C tightly coupled system (such as a super node), a UB-G loosely coupled distributed system, etc. The UB-C tightly coupled system refers to a RACK or a super node, and different computing nodes and host devices in the system are connected together through a high-speed network. The UB-G is a loosely coupled distributed system linked by UB, which supports peer-to-peer interconnection between nodes in the distributed system and can interwork with existing internet protocol (IP) and Ethernet (Eth) networks.

[0115] Referring to FIG. 8, in implementing the above data packet deduplication method, the hardware devices related to the embodiment of the present application include a forwarding device (such as a switch or a router) in the network and a receiving end device. The receiving end device can be an independent computing node or a certain network device, and the embodiment of the present application does not limit this.

[0116] It should be noted that for the forwarding device located between the sending end device and the receiving end device, each communication path can include one forwarding device or multiple forwarding devices. That is, the forwarding device can be directly connected with the receiving end device or connected with the receiving end device through at least one other forwarding device, and the embodiment of the present application does not limit this.

[0117] Please refer to FIG. 9, which is a structural schematic diagram of a network device provided in an embodiment of the present application. The network device can be a switch, a router or other device with a data packet forwarding function. In the embodiment of the present application, the network device serves as a data packet forwarding device to perform corresponding steps in the data packet deduplication method.

[0118] As shown in FIG. 9, in the embodiment of the present application, the network device includes a main control board 910, an interface board 930 and an interface board 940. In the case of multiple interface boards, a switching network board (not shown in FIG. 9) can be included to complete data switching between the interface boards (the interface board is also referred to as a line card or a service board).

[0119] The main control board 910 is used to complete system management, device maintenance, protocol processing and other functions. The interface boards 930 and 940 are used to provide various service interfaces (for example, a POS interface, a GE interface, an ATM interface and the like) and implement data flow forwarding. The main control board 910 mainly has three types of functional units: a system management control unit, a system clock unit and a system maintenance unit. The main control board 910, the interface board 930 and the interface board 940 are connected to a system backboard through a system bus to realize intercommunication. The interface board 930 includes one or more processors 931. The processor 931 is used to control and manage the interface board, communicate with a central processor on the main control board and implement data flow forwarding processing. A memory 932 on the interface board 930 is used to store forwarding table entries, and the processor 931 performs data flow forwarding by looking up the forwarding table entries stored in the memory 932.

[0120] The interface board 930 includes one or more network interfaces 933 used to receive data flows or other information sent by other devices and process the data flows or data according to the indication of the processor 931. The specific implementation process is not described here again.

[0121] It can be understood that, as shown in FIG. 9, the embodiment of the present application includes multiple interface boards and adopts a distributed forwarding mechanism. Under this mechanism, the operation on the interface board 940 is basically similar to the operation on the interface board 930. For the sake of brevity, the description is not repeated here. In addition, it can be understood that the processor 931 and / or 941 in the interface board 930 in FIG. 9 can be a special hardware or chip, such as a network processor or an application specific integrated circuit (ASIC), to implement the above functions. This implementation mode is the mode that the forwarding plane adopts a special hardware or chip processing. Of course, the processor 931 and / or 941 can also adopt a general-purpose processor, such as a general-purpose CPU, to implement the above-described functions.

[0122] In addition, it should be noted that the master control board can have one or more, and when there are multiple, it can include a main master control board and a backup master control board. The interface board can have one or more, and the stronger the data processing capability of the network device, the more interface boards are provided. In the case of multiple interface boards, the multiple interface boards can communicate through one or more switching network boards, and when there are multiple, they can jointly implement load sharing and redundancy backup. Under the centralized forwarding architecture, the network device can not need a switching network board, and the interface board undertakes the processing function of the entire system of service data. Under the distributed forwarding architecture, the network device includes multiple interface boards, and the data exchange between the multiple interface boards can be realized through the switching network board to provide large-capacity data exchange and processing capability. Therefore, the data access and processing capability of the network device of the distributed architecture is greater than that of the network device of the centralized architecture, and the specific architecture to be adopted depends on the specific networking deployment scenario, which is not limited here.

[0123] In some embodiments, the memory 932 can be a read-only memory (ROM), and can also be a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc including a compact disc read-only memory (CD-ROM), a compressed disc, a laser disc, a digital versatile disc, a Blu-ray disc, and the like, a magnetic disc storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory 932 can exist independently and be connected to the processor 931 through a communication bus. The memory 932 can also be integrated with the processor 931.

[0124] In some embodiments, the network interface 933 can be a device using any kind of transceiver, used for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc. The network interface 933 includes a wired network interface, and can also include a wireless network interface. The wired network interface can be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless network interface can be a WLAN interface, a cellular network communication interface, or a combination thereof, etc. When the network device is any network device in the domain, the network interface 933 is used to forward data packets to other network devices. When the network device is the head node in the domain, the network interface 933 can also be used to communicate with the receiving end device, such as sending data packets and deduplication trigger messages to the receiving end device.

[0125] In some embodiments, the network device can include a plurality of processors, each of which can be a single-core processor or a multi-core processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0126] In some embodiments, the memory 932 is used to store a computer program for implementing the solutions of the present application, and the processor 931 can execute the computer program stored in the memory 932 to cause the network device to perform the processing steps of the forwarding device in the method embodiments below. For specific implementation, reference can be made to the detailed description in the method embodiments, which will not be repeated here.

[0127] Please refer to FIG. 10, which is a structural schematic diagram of a computer device provided by an embodiment of the present application. The computer device can be a terminal device or a server. In the embodiments of the present application, the computer device serves as a receiving end device to perform the corresponding steps in the data packet deduplication method. As shown in FIG. 10, in the embodiments of the present application, the computer device includes at least one processor 101, a communication bus 102, a memory 103, and at least one communication interface 104.

[0128] The processor 101 can be a general central processing unit (CPU), a network processing unit (NP), a microprocessor, or can be one or more integrated circuits used to implement the schemes of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0129] The communication bus 102 is used to transmit information between the above-mentioned components. The communication bus 102 can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in FIG. 10, but it does not mean that there is only one bus or only one type of bus.

[0130] The memory 103 can be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disk (including a compact disc read-only memory (CD-ROM), a compact disc, a laser disc, a digital versatile disc, a Blu-ray disc, and the like), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto. The memory 103 can exist independently and be connected to the processor 101 through the communication bus 102. The memory 103 can also be integrated with the processor 101.

[0131] The communication interface 104 uses any transceiver-like mechanism for communicating with other devices or communication networks. The communication interface 104 includes a wired communication interface and can also include a wireless communication interface. The wired communication interface can be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface can be, for example, a wireless local area networks (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0132] As an example, the processor 101 can include one or more CPUs, such as CPU0 and CPU1 as shown in FIG. 10.

[0133] As an example, the computer device can include multiple processors, such as the processor 101 and the processor 105 as shown in FIG. 10. Each of these processors can be a single-core processor or a multi-core processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data, such as computer program instructions.

[0134] In some embodiments, the computer device can also include an output device and an input device. The output device is in communication with the processor 101 and can display information in various ways. For example, the output device can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device is in communication with the processor 101 and can receive user input in various ways. For example, the input device can be a mouse, a keyboard, a touch screen device, a sensor device, etc.

[0135] In some embodiments, the memory 103 is configured to store program code 110 for implementing the solutions of the present application, and the processor 101 can execute the program code 110 stored in the memory 103. The program code 110 can include one or more software modules, and the computer device can implement the processing steps of the receiving end device in the following method embodiments by means of the processor 101 and the program code 110 in the memory 103. The specific implementation can be referred to the detailed description in the following method embodiments, which will not be repeated here.

[0136] It should be noted that in some communication network architectures (for example, a Torus networking architecture), a computer device or a computing node can also serve as a forwarding device to forward a data packet, in which case the sending end device, the forwarding device, and the receiving end device can each be an independent computer device, such as a server, a terminal, a processor unit, a processor chip, and the like, and the present application does not limit this.

[0137] It should be understood that the network architecture, device architecture, and service scenario described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It can be known by those skilled in the art that, as the network architecture, device architecture evolve and new service scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0138] Next, the data packet deduplication method provided by the embodiments of the present application will be explained in detail in combination with the accompanying drawings.

[0139] Please refer to FIG. 11, which is a flowchart of a data packet deduplication method provided by an embodiment of the present application. The method is applied to a receiving end device, and the method includes the following steps.

[0140] Step 1101: receiving a first data packet.

[0141] For a data packet sent from a sending end device to a receiving end device, after the sending end device sends the data packet from the network, the data packet can be sequentially forwarded by multiple forwarding devices and finally forwarded to the receiving end device.

[0142] In the process of forwarding the data packet by multiple forwarding devices in the network, there can be a situation that some network devices fail or the links between network devices fail. At this time, in order to ensure that the receiving end device can normally receive the corresponding data packet, the first forwarding device that has forwarded the data packet will retransmit the data packet to the receiving end device along the backup path determined by the FRR based on the data packet backed up in the retransmission buffer of the first forwarding device. In this case, for the receiving end device, the first data packet it receives can be the data packet sent for the first time after the failure, or the data packet retransmitted after the failure.

[0143] In some embodiments, the forwarding device in the network can add a retransmission identifier in the data packet to indicate the receiving end device that the data packet is a retransmitted data packet.

[0144] As an example, referring to FIG. 12, in the UB-C domain, when the S1 switch detects that the communication path fails (for example, the path segment between the S1 switch and the S3 switch fails), the S1 switch determines a backup path through FRR, that is, the S1 switch-S2 switch-S3 switch-S4 switch-receiving end device, retransmits the data packet in the retransmission buffer, so that the data packet can be forwarded to the receiving end device along the backup path.

[0145] For the transaction layer of the data packet, the configuration (CFG) field value in the UB protocol is 6, indicating that the data packet is a transmission message in a certain family domain in the data center shown in FIG. 7.

[0146] In addition, before retransmitting the data packet, the S1 switch adds a retransmission flag bit "R" in the data packet header, sets the value of the retransmission flag bit "R" to 1, and then sends the data packet to the receiving end device through the backup path. After the receiving end device receives the data packet, the receiving end device parses the data packet header. If the value of "R" is 0, it is determined that the data packet is a first transmission data packet. If the value of "R" is 1, it is a retransmission data packet.

[0147] In some embodiments, when the forwarding device in the network detects that the target communication path for forwarding the first data packet fails, the forwarding device can also construct a first message and send the first message to the receiving end device to prompt the receiving end device that the data packet sent subsequently includes a retransmission data packet.

[0148] The first message includes a first field, and the first field carries a message type indicating that the first message is a deduplication trigger message used to instruct the receiving end device to perform deduplication processing on the data packet received after the first message.

[0149] As an example, as shown in FIG. 13, in the format of the complete compression network header under the UB compressed packet format, the first field can be a next layer protocol (NLP) field, and the specific value of the NLP field is used to indicate the message type. According to the different NLPs in the network header (NTH), there are three combination forms as follows:

[0150] The first form: NTH.NLP=000, with a transport packet header (TPH), and the TPH.NLP is used to indicate whether a confidentiality and integrity protection (CIP) mechanism is used.

[0151] Wherein, CIP.NLP indicates whether to carry a UB partition identifier (UB partition identifier, UPI) / entity ID (entity ID, referred to as EID, indicating the unique identifier of the UB entity in the UB domain) and its bit width.

[0152] The second: NTH.NLP = 001, without TPH, CIP, EID and transaction head (transaction head, TAH), with 16-bit UPI, used to identify the message for enumeration management protocol.

[0153] The third: NTH.NLP = 110, without TPH, CIP, UPI and TAH, with EID, used to identify the message as a deduplication trigger message.

[0154] Wherein, the carried EID can be the EID of the neighbor node connected by the fault node.

[0155] In a possible implementation, the first message can carry the network address (UB-C network address, CNA) of the sending end device in the UB-C domain as the source address information (denoted as SCNA), and carry the CNA of the receiving end device as the destination address information (denoted as DCNA).

[0156] In a possible implementation, the service priority (service level, SL) of the deduplication trigger message is set to the highest priority, so that the deduplication trigger message can enter the highest priority transmission queue, so as to be sent to the receiving end device as soon as possible.

[0157] It should be understood that in the complete compression network header shown in FIG. 13, the CRD (i.e., credit) indicates whether the message returns a letter of credit, ACK indicates whether the message releases the space of the retransmission buffer, and CRD_VL indicates the virtual link (VL) number corresponding to the credit returned by the CRD domain segment. Regarding the CFG domain segment, when CFG = 0, it indicates that the message is a data link layer control message; when CFG = 1-2, it is reserved; when CFG = 3, it indicates a Global_Domain-IPV4 message; when CFG = 4, it indicates a Global_Domain-IPV6 message; when CFG = 5, it indicates a network control message; when CFG = 6, it indicates a message in the family domain; and when CFG = 7-15, it is reserved. In addition, RM is the UB message routing mode indication domain segment. In addition, Plength[13:10] indicates the number of Blocks of the message, 0-15 respectively indicating 1-16 Blocks, Plength[9:5] indicating the number of Flits of the last Block, 0-31 respectively indicating 1-32 Flits. Plength[4:0] indicates the number of BYTEs of the data packet tail payload (excluding the LPH / LBH and BCRC domain segments). When Plength[4:0] is 0-15, it indicates that the number of payload BYTEs of the last Flit in the Packet is Plength[4:0]+1; when Plength[4:0] is 16-19, it indicates that the number of payload BYTEs of the second last Flit in the Packet is Plength[4:0]+1; when Plength[4:0] is 20-27, it indicates that the number of payload BYTEs of the second last Flit in the Packet is Plength[4:0]-11; and when Plength[4:0] is 28-31, it indicates that the number of payload BYTEs of the last Flit in the Packet is Plength[4:0]-11. The present embodiment does not make specific explanations and descriptions thereon, and the specific meaning can be referred to the related descriptions of the complete compression network header under the UB compressed message format, which will not be described herein.

[0158] Based on this, when the receiving end device executes the data packet deduplication method provided in the present embodiment, the receiving end device also needs to perform the following steps:

[0159] (1) receiving a first message.

[0160] The first message carries information of a forwarded data packet, the forwarded data packet is a data packet forwarded by a target forwarding device to a receiving device in a target fault detection period, and the first message is used to indicate that a fault is detected in the target communication path in the target fault detection period, and the target communication path is a communication path between the sending device and the receiving device.

[0161] The fault detection period is a pre-set value, and embodiments of the present application do not limit the setting basis thereof. For example, the fault detection period value can be flexibly set and adjusted based on the time (nanosecond level) at which a fault is found in network detection.

[0162] In a possible implementation, the target fault detection period is a period in which the receiving device receives the first data packet.

[0163] In some embodiments, the target communication path is a communication path between the sending device and the receiving device, and the target forwarding device can be at least one forwarding device on the communication path. That is, the target forwarding device can be one forwarding device that, when detecting a fault in a path segment connected to the forwarding device in the target communication path, constructs the first message and sends the first message to the receiving device. Of course, the target forwarding device can also be a plurality of forwarding devices that, when detecting a fault in a path segment connected to the forwarding device in the target communication path, construct the first message and send the first message to the receiving device. Embodiments of the present application do not limit this.

[0164] It should be noted that, for the target forwarding device, when detecting a fault in a path segment connected to the target forwarding device in the target communication path, the path segment can have a corresponding link fault, or the path segment can have a fault in another forwarding device connected thereto. When the path segment has a link fault, two forwarding devices connected to the link can detect the link fault, and at this time, the two forwarding devices detecting the link fault can construct the first message. When the path segment has a fault in a forwarding device connected thereto, another forwarding device connected to the forwarding device can detect the device fault, and at this time, one or more forwarding devices detecting the device fault can construct the first message.

[0165] It should be understood that, although the first message is constructed by the target forwarding device detecting the fault, when there are other forwarding devices between the target forwarding device and the receiving device, the target forwarding device can send the first message to the receiving device through the other forwarding devices. Of course, when the target forwarding device is directly connected to the receiving device, the target forwarding device can also directly send the first message to the receiving device. Embodiments of the present application do not limit the number of other forwarding devices between the target forwarding device and the receiving device.

[0166] In a possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device comprises at least one forwarding device detecting that the target device or the target path segment on the target communication path is faulty. If the target device is faulty, the target forwarding device comprises at least one forwarding device connected with the target device; if the target path segment is faulty, the target forwarding device comprises a first forwarding device and / or a second forwarding device, the second forwarding device being a next-hop forwarding device in communication connection with the first forwarding device, and the target path segment being a path segment between the first forwarding device and the second forwarding device.

[0167] The target forwarding device sends the first message to the receiving end device in the following two ways:

[0168] In the first sending way, the target forwarding device directly sends the first message to a specific receiving end device.

[0169] For ease of understanding, the implementation process of the target forwarding device sending the first message is described below in combination with the two network topologies shown in the foregoing.

[0170] Referring to FIG. 14, in the first network topology (2D Torus networking architecture), node 8 sends a data packet to node 7 along the main path, node 8 being a sending end device and node 7 being a receiving end device. In the process of sending the data packet, if a path segment between the sending end device and the receiving end device is faulty, for example, a link between node 9 and node 10 is faulty, node 9 and node 10 can construct a first message after detecting the link fault, and send the first message to node 7 along a determined backup path.

[0171] In which, the path for transmitting the data packet between node 8 and node 7 is node 8-node 9-node 10-node 11-node 7, and in the process of sending the first message, the transmission path adopted by node 9 is node 9-node 5-node 6-node 7, and the transmission path adopted by node 10 is node 10-node 11-node 7.

[0172] Of course, based on the actual network topology structure, node 8 and node 7 can also adopt other paths to transmit the data packet, and node 9 and node 10 can also adopt other paths to send the first message, which is not limited in the embodiments of the present application.

[0173] As can be seen, when a link corresponding to a path segment is faulty, a target forwarding device connected with the path segment can send a first message. The target forwarding device can be a previous forwarding device connected with the path segment, or a next forwarding device connected with the path segment, or both the previous and next forwarding devices connected with the path segment.

[0174] Referring to FIG. 15, in the first network topology (2D Torus networking architecture), node 8 sends a data packet to node 7 along a main path, node 8 is a sending device, and node 7 is a receiving device. During the sending of the data packet, if a forwarding device between the sending device and the receiving device fails, for example, node 9 fails, then node 5, node 8, node 10, and node 13 can construct a first message and send the first message to node 7 along a determined backup path after detecting the failure of node 9.

[0175] The path for transmitting the data packet between node 8 and node 7 is node 8-node 9-node 10-node 11-node 7, and the transmission path adopted by node 5 when sending the first message is node 5-node 6-node 7, the transmission path adopted by node 8 is node 8-node 4-node 7, the transmission path adopted by node 10 is node 10-node 11-node 7, and the transmission path adopted by node 13 is node 13-node 14-node 15-node 11-node 7.

[0176] Of course, based on the actual network topology, node 8 and node 7 can also use other paths to transmit the data packet, and node 5, node 8, node 10, and node 13 can also use other paths to send the first message, which is not limited by the embodiments of the application.

[0177] As can be seen, when a forwarding device connected to one end of a path segment fails, the first message can be sent by another forwarding device connected to the forwarding device.

[0178] Referring to FIG. 16, in the second network topology (Clos networking architecture), node A sends a data packet to node H along a main path, node A is a sending device, and node H is a receiving device. During the sending of the data packet, if a path segment between the sending device and the receiving device fails, for example, a link between node Leaf 1 and node Spine 1 fails, then node Leaf 1 and node Spine 1 can construct a first message and send the first message to node H along a determined backup path after detecting the failure of the link.

[0179] The path for transmitting the data packet between node A and node H is node A-node Leaf 1-node Spine 1-node Leaf n-node H, and the transmission path adopted by node Leaf 1 when sending the first message is node Leaf 1-node Spine 16-node Leaf n-node H, and the transmission path adopted by node Spine 1 is node Spine 1-node Leaf n-node H.

[0180] Of course, based on the actual network topology, node A and node H can also use other paths to transmit the data packet, and node Leaf 1 can also use other paths to send the first message, and the embodiments of the application do not limit this.

[0181] As can be seen, when the link corresponding to the path segment fails, the first message can be sent by the target forwarding device connected by the path segment. The target forwarding device can be the previous forwarding device connected by the path segment, or the next forwarding device connected by the path segment, or the two forwarding devices connected by the path segment.

[0182] Referring to FIG. 17, in the second network topology (Clos network architecture), node A sends a data packet to node H along the main path, node A is a sending end device, and node H is a receiving end device. During the sending of the data packet, if there is a fault in the forwarding device between the sending end device and the receiving end device, for example, node Spine 1 has a fault, then node Leaf 1 and node Leaf n can construct a first message after detecting the link fault, and send the first message to node H along the determined backup path.

[0183] The path for transmitting the data packet between node A and node H is node A-node Leaf 1-node Spine 1-node Leaf n-node H, and when the first message is sent, the transmission path used by node Leaf 1 is node Leaf 1-node Spine 16-node Leaf n-node H, and the transmission path used by node Leaf n is node Leaf n-node H.

[0184] Of course, based on the actual network topology, node A and node H can also use other paths to transmit the data packet, and node Leaf 1 can also use other paths to send the first message, and the embodiments of the application do not limit this.

[0185] As can be seen, when the link corresponding to the path segment fails, the first message can be sent by the target forwarding device connected by the path segment. The target forwarding device can be the previous forwarding device connected by the path segment, or the next forwarding device connected by the path segment, or the two forwarding devices connected by the path segment.

[0186] It should be noted that when the target forwarding device sends the first message to the specified receiving end device, the device information of the receiving end device needs to be carried in the first message for the next-hop forwarding device to forward the message. The device information can be the device type, device number, device address, or other information that can uniquely indicate the receiving end device.

[0187] In a possible implementation, the target forwarding device can maintain the DCNA information of the data packets forwarded in a T time period, and carry the DCNA information of the receiving end device in the first message when sending the first message to the specified receiving end device.

[0188] The value of T can be equal to the value of the fault detection period or greater than the value of the fault detection period, and the embodiments of the present application do not limit this.

[0189] In the second sending mode, the target forwarding device sends the first message to the specific receiving end device in a broadcast manner.

[0190] Similarly, for ease of understanding, the implementation process of the target forwarding device sending the first message will be illustrated in combination with the two network topologies shown above.

[0191] With reference to FIG. 14, if the node 10 detects a link fault, the node 10 constructs a first message and sends the first message to the node 7 in a broadcast manner. The node 10 can send the first message to the node 7 through at least two-hop broadcast and send the first message to other nodes in the first network topology through at least one-hop broadcast.

[0192] For example, the node 10 sends the first message to the node 11, the node 6 and the node 14 through one-hop broadcast, sends the first message to the node 13, the node 14, the node 15, the node 8, the node 7, the node 5 and the node 2 through two-hop broadcast, and sends the first message to the node 12, the node 4, the node 1, the node 0 and the node 3 through three-hop broadcast.

[0193] Of course, the node 9 can also broadcast the first message to other nodes according to the above logic after detecting a link fault, and the embodiments of the present application do not limit this.

[0194] With reference to FIG. 15, if the node 8 and the node 10 detect a node 9 fault, the node 8 and the node 10 construct a first message and send the first message to the node 7 in a broadcast manner. The node 8 and the node 10 can send the first message to the node 7 through at least two-hop broadcast and send the first message to other nodes in the first network topology through at least one-hop broadcast.

[0195] For example, the node 10 sends the first message to the node 11, the node 6 and the node 14 through one-hop broadcast, to the node 13, the node 14, the node 15, the node 8, the node 7, the node 5, the node 2 through two-hop broadcast, and to the node 12, the node 4, the node 1, the node 0, the node 3 through three-hop broadcast. The node 8 sends the first message to the node 12, the node 4 and the node 11 through one-hop broadcast, to the node 13, the node 5, the node 0, the node 12 through two-hop broadcast, and to the node 1, the node 3 through three-hop.

[0196] Of course, the node 5 and the node 13 can also broadcast the first message to other nodes according to the above logic after detecting the node 9 failure, which is not limited by the embodiments of the application.

[0197] With reference back to FIG. 16, if the node Spine 1 detects the link failure, the node Spine 1 constructs the first message and sends the first message to the node H through broadcast. The node Spine 1 can send the first message to the node H through at least two-hop broadcast and to other nodes in the second network topology through at least one-hop broadcast.

[0198] For example, the node Spine 1 sends the first message to the node Leaf 2, the node Leaf n-1, the node Leaf n through one-hop broadcast, to the node C, the node D, the node E, the node F, the node G, the node H through two-hop broadcast.

[0199] Of course, the node Leaf 1 can also broadcast the first message to other nodes (for example, the node A and the node B) according to the above logic after detecting the link failure, which is not limited by the embodiments of the application.

[0200] With reference back to FIG. 17, if the node Leaf 1 detects the node Spine 1 failure, the node Leaf 1 constructs the first message and sends the first message to the node H through broadcast. The node Leaf 1 can send the first message to the node H through at least three-hop broadcast and to other nodes in the second network topology through at least one-hop broadcast.

[0201] For example, the node Leaf 1 sends the first message to the node Spine 16, the node A and the node B through one-hop broadcast, to Leaf 2, Leaf n-1, Leaf n through two-hop broadcast, and to the node C, the node D, the node E, the node F, the node G, the node H through three-hop broadcast.

[0202] Of course, the nodes Leaf 1, Leaf 2, Leaf n-1, Leaf n can also broadcast the first message to other nodes according to the above logic after detecting the failure of the node Spine 1, and the embodiments of the present application do not limit this.

[0203] In some embodiments, regardless of the sending mode used to send the first message, the target forwarding device can also set the SL of the message to the highest priority when sending the first message, and then send it to the receiving end device.

[0204] In summary, after introducing the way in which the target forwarding device sends the first message in combination with FIG. 14-FIG. 17, the information carried by the first message will be explained in detail.

[0205] As explained above, the first message sent to the receiving end device carries the information of the forwarded data packet, which is the data packet forwarded by the target forwarding device to the receiving end device within the target failure detection period.

[0206] In some embodiments, the information of the forwarded data packet includes identification information of the forwarded data packet. The identification information includes a transaction layer segment sequence number (TASSN) of the forwarded data packet or packet sequence number (PSN) information of the forwarded data packet.

[0207] Based on this, the first message includes a first field for carrying the identification information of the forwarded data packet.

[0208] In a possible implementation, the information of the forwarded data packet further includes device address information, which at least indicates the address of the sending end device.

[0209] Based on this, the first message includes a first field and a second field, the first field is used to carry the identification information of the forwarded data packet, and the second field is used to carry the device address information.

[0210] In a possible implementation, the first message can also carry other information of the data packet, such as SDNA, DCNA, TPH, TAH, etc.

[0211] As an example, as shown in FIG. 18, the first message carries the SCNA and TASSN corresponding to the data packet forwarded by the target forwarding device to the receiving end device within the target failure detection period.

[0212] As another example, as shown in FIG. 19, the first message carries the SCNA and PSN corresponding to the data packet forwarded by the target forwarding device to the receiving end device within the target failure detection period.

[0213] wherein, SCNA is the address information of the sending device in the UB-C domain. In this way, the receiving device can determine the data packet from which sending device based on the SCNA.

[0214] It should be noted that the meaning of other fields in the first message and the content carried can refer to the related description of the complete compression network header under the UB compression message format, and the embodiments of the present application will not be repeated here.

[0215] (2) Based on the data packet information, update the deduplication matching table.

[0216] That is, based on the first message, the information carried therein that the target forwarding device has forwarded the data packet is added to the deduplication matching table, and the update of the deduplication matching table has been realized.

[0217] Based on the two types of information carried in the above first message, the deduplication matching table maintained by the receiving device in the embodiments of the present application also includes the following two cases:

[0218] The first case, see Table 1 below, based on the SCNA and TASSN carried in the first message, the information stored in the deduplication matching table maintained and managed by the receiving device includes: the SCNA and TASSN corresponding to the data packet that the target forwarding device has forwarded to the receiving device. That is, the deduplication matching table records the correspondence between the SCNA and TASSN of the data packet.

[0219] Table 1

[0220] In a possible implementation, the deduplication matching table can also store the correspondence between the Hash values of SCNA and TASSN.

[0221] The second case, see Table 2 below, based on the SCNA and PSN carried in the first message, the information stored in the deduplication matching table maintained and managed by the receiving device includes: the SCNA, EPSN and maximum out-of-order space corresponding to the data packet that the target forwarding device has forwarded to the receiving device. That is, the deduplication matching table records the correspondence between the SCNA, EPSN and maximum out-of-order space of the data packet.

[0222] Table 2

[0223] It should be noted that the maximum allowed out-of-order space indicates the number of out-of-order packets allowed to be received before the expected in-order data packet, which is not shown in the above Table 2.

[0224] Step 1102: If the first data packet carries the retransmission identifier, performing deduplication processing on the first data packet according to the deduplication matching table, so that there is one first data packet in the receiving end device, and the deduplication matching table stores information of data packets received by the receiving end device.

[0225] In some embodiments, when the forwarding device needs to mark the retransmitted data packet, the retransmission identifier can be carried in the reserved bit in the data packet header.

[0226] As an example, as shown in FIG. 20, the “4” reserved bit of Byte1 in the data packet header is defined as a retransmission identifier bit (denoted as R) to identify whether the data packet is a retransmitted data packet. When the value of the retransmission identifier bit is R=1, it indicates that the first data packet is a data packet taken out from the retransmission buffer and retransmitted by the target forwarding device again.

[0227] In some embodiments, the first data packet also carries an end retransmission identifier, which indicates that the first data packet is the last data packet for deduplication processing.

[0228] As an example, continuing to refer to FIG. 20, the “1” reserved bit of Byte0 in the data packet header can be defined as an end retransmission identifier bit (denoted as LR), and the information of the bit is used to indicate whether the first data packet is the last retransmitted data packet. When the value of the retransmission identifier bit is R=1 and the value of the end retransmission identifier bit is LR=1, it indicates that the first data packet is the last retransmitted data packet taken out from the retransmission buffer.

[0229] In a possible implementation, the implementation process of step 1102 can be: if it is determined according to the deduplication matching table that the first data packet is a duplicate data packet, discarding the first data packet; and if it is determined according to the deduplication matching table that the first data packet is a non-duplicate data packet, retaining the first data packet.

[0230] Further, if the first data packet is a non-duplicate data packet, after retaining the first data packet, the information of the first data packet also needs to be recorded in the deduplication matching table to update the deduplication matching table.

[0231] In a possible implementation, if the first data packet does not carry the retransmission identifier, the first data packet is not subjected to deduplication processing. That is, in the case that the first data packet does not carry the retransmission identifier, the first data packet is processed by using normal data processing logic, for example, sending the first data packet, storing the first data packet, etc., which is not limited in the embodiments of the present application.

[0232] As an example, referring to FIG. 21, when the data packet pkt1 does not carry the retransmission identifier, the data packet pkt1 is directly sent up; for the data packet pkt2, if the information of the data packet pkt2 is carried in the de-duplication triggering message, after receiving the data packet pkt2, the de-duplication matching table is queried based on the information of the data packet pkt2 to determine whether the data packet pkt2 is a duplicate data packet, if the data packet pkt2 is not a duplicate data packet, the data packet pkt2 is sent up, and the de-duplication matching table is updated based on the information of the data packet pkt2. For the received data packet pkt3, if the data packet pkt3 carries the retransmission identifier, the de-duplication matching table is queried based on the data packet pkt3 to determine whether the data packet pkt3 is a duplicate data packet, if the data packet pkt3 is not a duplicate data packet, the data packet pkt3 is sent up, and the de-duplication matching table is updated based on the information of the data packet pkt3.

[0233] In summary, in the embodiment of the present application, after the receiving end device receives a data packet, if the data packet carries the retransmission identifier, it indicates that the data packet is a retransmitted data packet in the network, and for the retransmitted data packet, the receiving end device may have received the same data packet before, therefore, in the case that the data packet carries the retransmission identifier, the receiving end device needs to perform de-duplication processing on the data packet. In this way, it can be ensured that there is no duplicate data packet in the receiving end device, so that the receiving end device can process the network forwarded data packet exactly once.

[0234] Moreover, compared with the scheme in the related art that needs to judge whether each data packet is a duplicate data packet according to the PSN carried by the data packet when detecting the duplicate packet based on the PSN, in the embodiment of the present application, the receiving end device can directly judge whether the data packet is a retransmitted data packet according to whether the data packet carries the retransmission identifier in the process of receiving the data packet, and then only needs to perform de-duplication processing on a small number of data packets marked with the retransmission identifier based on the de-duplication matching table, without performing the operation of judging whether each data packet is a duplicate data packet, which greatly reduces the resource consumption of the receiving end device, and implements the lightweight de-duplication of the receiving end device on the data packet.

[0235] Based on the data packet de-duplication method shown in the above embodiment, the data packet de-duplication method provided by the embodiment of the present application is explained and described through the interaction between the target forwarding device and the receiving end device, and the data packet de-duplication method includes the following steps S1-S6:

[0236] Step S1: The target forwarding device sends a first data packet to the receiving end device.

[0237] Step S2: The receiving end device receives the first data packet.

[0238] Step S3: If the first data packet carries the retransmission identifier, the receiving end device performs deduplication processing on the first data packet according to the deduplication matching table, so that there is one first data packet in the receiving end device.

[0239] The deduplication matching table stores information of data packets received by the receiving end device.

[0240] In a possible implementation, the implementation process of performing deduplication processing on the first data packet according to the deduplication matching table can be as follows: if it is determined according to the deduplication matching table that the first data packet is a duplicate data packet, the first data packet is discarded; if it is determined according to the deduplication matching table that the first data packet is a non-duplicate data packet, the first data packet is retained.

[0241] The retention of the first data packet means that the first data packet is stored, and information of the first data packet is recorded in the deduplication matching table, so as to update the deduplication matching table.

[0242] Step S4: If the first data packet does not carry the retransmission identifier, the receiving end device processes the first data packet according to normal data processing logic.

[0243] It should be noted that the first data packet carries the retransmission identifier because the target forwarding device detects that the target communication path is faulty and performs FRR, in which case, in order to ensure reliable transmission of the data packet, the target forwarding device marks the retransmitted first data packet, so that the first data packet carries the retransmission identifier.

[0244] The target communication path is a communication path between the sending end device and the receiving end device.

[0245] In a possible implementation, the first data packet further carries an end retransmission identifier, which indicates that the first data packet is the last data packet subjected to deduplication processing.

[0246] That is, in the case where the target forwarding device needs to retransmit a plurality of data packets to the receiving end device, the target forwarding device can mark the plurality of data packets with the retransmission identifier, to indicate that the plurality of data packets are retransmitted data packets; and mark the last data packet in the plurality of data packets with the end retransmission identifier, to indicate that the data packet is the last retransmitted data packet.

[0247] In the embodiment of the application, the end retransmission identifier can instruct the receiving end device to end the deduplication processing operation, that is, not to perform deduplication processing on data packets received after the data packet carrying the end retransmission identifier.

[0248] Further, in a case where the target forwarding device detects that the target communication path is faulty, the target forwarding device also sends the first message to the receiving end device to prompt the receiving end device that there may be retransmission data packets in the future. Based on this, the data packet deduplication method provided by the embodiments of the present application further includes the following steps:

[0249] Step S5: The target forwarding device sends a first message to the receiving end device, and the first message carries information of the forwarded data packet, the forwarded data packet being a data packet forwarded by the target forwarding device to the receiving end device within the target fault detection period.

[0250] The first message includes a first field, and the first field carries a message type, the message type indicating that the first message is a deduplication trigger message, the deduplication trigger message being used to instruct the receiving end device to perform deduplication processing on a data packet received after the first message.

[0251] In a specific implementation, the first message is used to indicate that the target communication path is detected to be faulty within the target fault detection period, and the target communication path is a communication path between the sending end device and the receiving end device.

[0252] In this case, the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects that a target device or a target path segment on the target communication path is faulty. If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; if the target path segment is faulty, the target forwarding device includes a first forwarding device and / or a second forwarding device, the second forwarding device being a next-hop forwarding device on the target communication path that is communicatively connected to the first forwarding device, and the target path segment being a path segment between the first forwarding device and the second forwarding device.

[0253] In a possible implementation, the information of the forwarded data packet includes identification information of the forwarded data packet, and the identification information includes a transaction layer segment number of the forwarded data packet or packet sequence number information of the forwarded data packet.

[0254] In a possible implementation, the information of the forwarded data packet further includes device address information, and the device address information at least indicates an address of the sending end device.

[0255] Step S6: The receiving end device receives the first message, and updates the deduplication matching table based on the information of the forwarded data packet.

[0256] Next, the implementation process of the target forwarding device marking a data packet with a retransmission identifier, the target forwarding device sending a first message to the receiving end device, and the receiving end device implementing lightweight deduplication will be described by way of example with reference to the relevant drawings, to supplement the description of the specific implementation process.

[0257] With reference to the foregoing Fig. 12, in the process that the sending end device sends the data packet to the receiving end device through the main path, i.e., S1 switch-S3 switch-S4 switch-receiving end device, if the S1 switch and the S3 switch detect a link fault, the first message can be constructed at the first time, the SL level of the message is set to the highest priority, and the first message is sent to the receiving end device.

[0258] Taking the S3 switch as an example, there are two ways for the S3 switch to send the retrigger message:

[0259] (1) The S3 switch maintains the DCNA information of the forwarded data packet in a T time period, at this time, the first message can be directly sent to the receiving end device through the standby path based on the DCNA information.

[0260] (2) The S3 switch sends the first message in a broadcast manner.

[0261] The standby path is S1 switch-S2 switch-S3 switch-S4 switch-receiving end device.

[0262] With reference to Fig. 22, for the S2 switch, when it receives the data packet, the internal logic for processing the data packet is as follows: after receiving the Block / Flit of the data packet from a certain port, if the data packet is correct or the decoding is successful, the data packet is sent to the service support unit (SSU), and the data packet is accumulated after all Block / Flits of the data packet are received, and the data packet is forwarded from a certain port based on the DCNA information in the LPH, and the backup of the data packet is saved to the retransmission buffer.

[0263] For the S2 switch, after receiving the ACK message sent by the S3 switch of the next hop through the LPH / LBH, the corresponding space in the retransmission buffer is released. In addition, the information of the data packet (such as some or all data packet header information, or a Hash value) is saved to the buffer to facilitate the construction of the first message.

[0264] In the embodiment of the present application, for the switches in the network, the retransmission buffer needs to be managed at the data packet level. If part of the Block / Flit of a data packet does not receive an ACK message, the corresponding interval resource of the data packet in the retransmission buffer is not released temporarily.

[0265] As shown in FIG. 23, for a packet (Packet, pkt for short) including Addr3-Addr6, a total of four Blocks / Flits. If Addr5 in it does not receive an ACK message, the switch retransmits Addr3-Addr6 through a backup path after FRR, and temporarily does not release the interval resource corresponding to the packet.

[0266] Further, after receiving the packet, the receiving end device parses the packet header. As shown in FIG. 24, if the receiving end device parses the packet header and determines that the message type carried in the first field in the packet is a deduplication trigger message, starts the deduplication module of the receiving end device, and updates the deduplication matching table based on the information carried in the first message. As shown in FIG. 25, if the receiving end device parses the packet header and determines that the packet carries a retransmission identifier, it is necessary to query the deduplication matching table based on the information of the packet to detect whether the packet is a duplicate packet. If the packet is a duplicate packet, the packet is discarded; if it is not a duplicate packet, the deduplication matching table is updated based on the information of the packet, and the packet is uploaded to the upper layer module.

[0267] As can be seen, for the packet forwarded by the network, if the packet is a packet retransmitted from the retransmission buffer, the packet carries a retransmission identifier. In other words, for the forwarding device in the network, a small amount of retransmitted packets are marked when forwarding the packets to the receiving end device, so that they carry a retransmission identifier.

[0268] For the receiving end device, after receiving a certain packet, if the packet carries a retransmission identifier, it indicates that the packet is a retransmitted packet in the network. For the retransmitted packet, the receiving end device may have received the same packet before, so in the case where the packet carries a retransmission identifier, the receiving end device needs to perform deduplication processing on the packet. In this way, it can be ensured that there is no duplicate packet in the receiving end device, so that the receiving end device can process the packet forwarded by the network exactly once.

[0269] Moreover, compared with the scheme in the related art that detects duplicate packets based on PSN, which needs to judge whether each packet is a duplicate packet according to the PSN carried by the packet, the receiving end device in the embodiment of the present application can directly judge whether the packet is a retransmitted packet according to whether the packet carries a retransmission identifier in the process of receiving the packet, and then only needs to perform deduplication processing on a small amount of packets marked with a retransmission identifier based on the deduplication matching table, without performing the operation of judging whether each packet is a duplicate packet, greatly reducing the resource consumption of the receiving end device, and realizing lightweight deduplication of the receiving end device on the packet.

[0270] It should be noted that the related content of FIG. 22-FIG. 25 can also refer to the above-mentioned explanation of the method embodiment of FIG. 11, and will not be repeated here.

[0271] FIG. 26 is a structural schematic diagram of a data packet deduplication device provided by an embodiment of the application. The data packet deduplication device can be realized by software, hardware, or a combination of both, as part of or all of a receiving end device. Referring to FIG. 26, the data packet deduplication device 2600 includes a first receiving module 2601 and a deduplication module 2602.

[0272] The first receiving module 2601 is configured to receive a first data packet.

[0273] The deduplication module 2602 is configured to, if the first data packet carries a retransmission identifier, perform deduplication processing on the first data packet according to a deduplication matching table, so that there is only one first data packet in the receiving end device, and the deduplication matching table stores information of data packets that have been received by the receiving end device.

[0274] In a possible implementation, the deduplication module 2602 is specifically configured to:

[0275] If it is determined according to the deduplication matching table that the first data packet is a duplicate data packet, the first data packet is discarded.

[0276] If it is determined according to the deduplication matching table that the first data packet is a non-duplicate data packet, the first data packet is retained.

[0277] In a possible implementation, the data packet deduplication device 2600 further includes:

[0278] An updating module configured to store the first data packet, and record information of the first data packet in the deduplication matching table, so as to update the deduplication matching table.

[0279] In a possible implementation, the data packet deduplication device 2600 further includes:

[0280] The second receiving module is further configured to receive a first message, the first message carrying information of a forwarded data packet, the forwarded data packet being a data packet forwarded by a target forwarding device to the receiving end device within a target fault detection period, the first message being used to indicate that a fault is detected in a target communication path within the target fault detection period, the target communication path being a communication path between a sending end device and the receiving end device.

[0281] The updating module is configured to update the deduplication matching table based on the information of the forwarded data packet.

[0282] In a possible implementation, the first message comprises a first field, and the first field carries a message type, and the message type indicates that the first message is a deduplication trigger message, and the deduplication trigger message is used to instruct the receiving end device to perform deduplication processing on a data packet received after the first message.

[0283] In a possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device comprises at least one forwarding device that detects that the target device or the target path segment on the target communication path is faulty;

[0284] If the target device is faulty, the target forwarding device comprises at least one forwarding device connected with the target device;

[0285] If the target path segment is faulty, the target forwarding device comprises a first forwarding device and / or a second forwarding device, the second forwarding device is a next-hop forwarding device in the target communication path and is communicatively connected with the first forwarding device, and the target path segment is a path segment between the first forwarding device and the second forwarding device.

[0286] In a possible implementation, the information of the forwarded data packet comprises identification information of the forwarded data packet.

[0287] The identification information comprises a transaction layer segment number of the forwarded data packet, or packet sequence number information of the forwarded data packet.

[0288] In a possible implementation, the deduplication trigger message comprises a first field, and the first field is used to carry the identification information of the forwarded data packet.

[0289] In a possible implementation, the information of the forwarded data packet further comprises device address information, and the device address information at least indicates an address of the sending end device.

[0290] In a possible implementation, the deduplication trigger message comprises a first field and a second field, the first field is used to carry the identification information of the forwarded data packet, and the second field is used to carry the device address information.

[0291] In a possible implementation, the first data packet further carries an end-retransmission identifier, and the end-retransmission identifier indicates that the first data packet is the last data packet for deduplication processing.

[0292] In the embodiments of the present application, for the received data packet, if the data packet carries the retransmission identifier, it indicates that the data packet is a retransmitted data packet in the network, and for the retransmitted data packet, the receiving end device may have received the same data packet before, and therefore, in the case that the data packet carries the retransmission identifier, the receiving end device needs to perform the deduplication processing on the data packet. In this way, it can be ensured that there is no duplicate data packet in the receiving end device, so that the receiving end device can process the data packet forwarded by the network exactly once.

[0293] Moreover, compared with the scheme in the related art that needs to judge whether each data packet is a duplicate data packet according to the PSN carried by the data packet when detecting the duplicate packet based on the PSN, in the embodiments of the present application, the receiving end device can directly judge whether the data packet is a retransmitted data packet according to whether the data packet carries the retransmission identifier in the process of receiving the data packet, and then only needs to perform the deduplication processing based on the deduplication matching table for a small number of data packets marked with the retransmission identifier, without performing the operation of judging whether each data packet is a duplicate data packet, which greatly reduces the resource consumption of the receiving end device and realizes the lightweight deduplication of the receiving end device on the data packet.

[0294] It should be noted that: the data packet deduplication apparatus 2600 provided in the above embodiments only takes the division of the above functional modules as an example for illustrating the deduplication processing on the received data packet, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the data packet deduplication apparatus 2600 provided in the above embodiments and the data packet deduplication method embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0295] FIG. 27 is a structural schematic diagram of another data packet deduplication apparatus provided in the embodiments of the present application. The data packet deduplication apparatus can be realized by software, hardware or a combination of both to become part or all of the target forwarding device. Referring to FIG. 27, the data packet deduplication apparatus 2700 includes a first sending module 2701.

[0296] The first sending module 2701 is configured to send a first data packet to a receiving end device, to instruct the receiving end device to perform deduplication processing on the first data packet according to a deduplication matching table, so that there is one first data packet in the receiving end device; wherein the first data packet carries a retransmission identifier, and the deduplication matching table stores information of data packets that have been received by the receiving end device.

[0297] In a possible implementation manner, the data packet deduplication apparatus 2700 further includes:

[0298] The second sending module is configured to send a first message to the receiving end device, so as to instruct the receiving end device to update the deduplication matching table based on information of a forwarded data packet carried in the first message;

[0299] The forwarded data packet is a data packet forwarded by a target forwarding device to the receiving end device within a target fault detection period, and the first message is used to indicate that a fault is detected in the target communication path within the target fault detection period, and the target communication path is a communication path between the sending end device and the receiving end device.

[0300] In a possible implementation, the first message comprises a first field, and the first field carries a message type, and the message type indicates that the first message is a deduplication trigger message, and the deduplication trigger message is used to instruct the receiving end device to perform deduplication processing on a data packet received after the first message.

[0301] In a possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device comprises at least one forwarding device that detects a fault of a target device or a target path segment on the target communication path;

[0302] If the target device has a fault, the target forwarding device comprises at least one forwarding device connected with the target device;

[0303] If the target path segment has a fault, the target forwarding device comprises a first forwarding device and / or a second forwarding device, the second forwarding device is a next-hop forwarding device in the target communication path and is communicatively connected with the first forwarding device, and the target path segment is a path segment between the first forwarding device and the second forwarding device.

[0304] In a possible implementation, the information of the forwarded data packet comprises identification information of the forwarded data packet, and the identification information comprises a transaction layer segment number of the forwarded data packet or packet sequence number information of the forwarded data packet.

[0305] In a possible implementation, the information of the forwarded data packet further comprises device address information, and the device address information at least indicates an address of the sending end device.

[0306] In a possible implementation, the first data packet further carries an end retransmission identifier, and the end retransmission identifier indicates that the first data packet is the last data packet for deduplication processing.

[0307] In the embodiments of the present application, for a data packet forwarded by a network, if the data packet is a data packet retransmitted from a retransmission buffer, the data packet carries a retransmission identifier. In other words, for a forwarding device in the network, when the forwarding device forwards a data packet to a receiving end device, the forwarding device marks a small amount of retransmitted data packets so that the retransmitted data packets carry retransmission identifiers.

[0308] Further, compared with the scheme of judging whether each data packet is a duplicate data packet according to the PSN carried by the data packet when detecting duplicate packets based on the PSN in the related art, since the forwarding device in the embodiment of the present application can mark a small amount of retransmitted data packets, the receiving end device can directly judge whether the data packet is a retransmitted data packet according to whether the data packet carries a retransmission identifier, and then only needs to perform deduplication processing based on the deduplication matching table for a small amount of data packets marked with the retransmission identifier, without performing the operation of judging whether each data packet is a duplicate data packet, which greatly reduces the resource consumption of the receiving end device and implements lightweight deduplication of the data packet by the receiving end device.

[0309] It should be noted that: the data packet deduplication device 2700 provided by the above embodiment is only exemplified by the division of the above functional modules when marking the retransmitted data packet and sending the first message, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the data packet deduplication device 2700 provided by the above embodiment and the data packet deduplication method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0310] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, when the computer program runs on a computer or a processor, makes the computer or the processor execute the steps of the data packet deduplication method shown in the above embodiment.

[0311] The embodiment of the present application also provides a computer program product, which contains computer instructions, when the computer instructions are executed by a computer or a processor, makes the computer or the processor execute the steps of the data packet deduplication method shown in the above embodiment. Or, a computer program is provided, when the computer program runs on a computer or a processor, makes the computer or the processor execute the steps of the data packet deduplication method shown in the above embodiment.

[0312] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (for example: coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example: infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example: floppy disk, hard disk, magnetic tape), optical media (for example: digital versatile disc (DVD)) or semiconductor media (for example: solid state disk (SSD)) and the like. It should be noted that the computer-readable storage medium mentioned in the embodiments of the present application can be a non-volatile storage medium, in other words, it can be a non-transitory storage medium.

[0313] It should be understood that "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; "and / or" herein only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, "first", "second" and the like are used to distinguish the same items or similar items with basically the same function and role in the embodiments of the present application. Those skilled in the art can understand that "first", "second" and the like do not limit the quantity and execution order, and "first", "second" and the like do not necessarily mean different.

[0314] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0315] The above describes the embodiments provided by the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data packet deduplication method, characterized in that, Applied to a receiving device, the method includes: Receive the first data packet; If the first data packet carries a retransmission flag, the first data packet is deduplicated according to the deduplication matching table so that there is a first data packet in the receiving device. The deduplication matching table stores information about the data packets that the receiving device has received.

2. The method as described in claim 1, characterized in that, The step of deduplicating the first data packet according to the deduplication matching table includes: If the first data packet is determined to be a duplicate data packet according to the deduplication matching table, then the first data packet is discarded; If the first data packet is determined to be a non-duplicate data packet according to the deduplication matching table, then the first data packet is retained.

3. The method as described in claim 2, characterized in that, The retention of the first data packet includes: The first data packet is stored, and its information is recorded in the deduplication matching table to update the deduplication matching table.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: A first message is received, the first message carrying information about a forwarded data packet, the forwarded data packet being a data packet forwarded by the target forwarding device to the receiving device within the target fault detection period, the first message being used to indicate that a fault is detected in the target communication path within the target fault detection period, the target communication path being the communication path between the sending device and the receiving device; The deduplication matching table is updated based on the information of the forwarded data packets.

5. The method as described in claim 4, characterized in that, The first message includes a first field, which carries a message type. The message type indicates that the first message is a deduplication trigger message. The deduplication trigger message is used to instruct the receiving device to perform deduplication processing on data packets received after the first message.

6. The method as described in claim 4 or 5, characterized in that, The target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path; If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; If the target path segment is faulty, the target forwarding device includes a first forwarding device and / or a second forwarding device, wherein the second forwarding device is the next-hop forwarding device in the target communication path that is connected to the first forwarding device, and the target path segment is the path segment between the first forwarding device and the second forwarding device.

7. The method according to any one of claims 4-6, characterized in that, The information of the forwarded data packet includes the identification information of the forwarded data packet, which includes the transaction layer segment number of the forwarded data packet or the packet sequence number information of the forwarded data packet.

8. The method as described in claim 7, characterized in that, The information in the forwarded data packet also includes device address information, which at least indicates the address of the sending device.

9. The method according to any one of claims 1-8, characterized in that, The first data packet also carries an end-of-retransmission flag, which indicates that the first data packet is the last data packet to undergo deduplication.

10. A method for deduplicating data packets, characterized in that, Applied to a target forwarding device, the method includes: A first data packet is sent to the receiving device to instruct the receiving device to perform deduplication processing on the first data packet according to the deduplication matching table, so that there is a first data packet in the receiving device; wherein, the first data packet carries a retransmission identifier, and the deduplication matching table stores information on data packets that the receiving device has received.

11. The method as described in claim 10, characterized in that, The method further includes: Send a first message to the receiving device to instruct the receiving device to update the deduplication matching table based on the information of the forwarded data packets carried in the first message; Wherein, the forwarded data packet is the data packet forwarded by the target forwarding device to the receiving device within the target fault detection period, and the first message is used to indicate that a fault is detected in the target communication path within the target fault detection period, and the target communication path is the communication path between the sending device and the receiving device.

12. The method as described in claim 11, characterized in that, The first message includes a first field, which carries a message type. The message type indicates that the first message is a deduplication trigger message. The deduplication trigger message is used to instruct the receiving device to perform deduplication processing on data packets received after the first message.

13. The method according to any one of claims 10-12, characterized in that, The target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path; If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; If the target path segment is faulty, the target forwarding device includes a first forwarding device and / or a second forwarding device, wherein the second forwarding device is the next-hop forwarding device in the target communication path that is connected to the first forwarding device, and the target path segment is the path segment between the first forwarding device and the second forwarding device.

14. The method as described in claim 11, characterized in that, The information of the forwarded data packet includes the identification information of the forwarded data packet, which includes the transaction layer segment number of the forwarded data packet or the packet sequence number information of the forwarded data packet.

15. The method as described in claim 14, characterized in that, The information in the forwarded data packet also includes device address information, which at least indicates the address of the sending device.

16. The method according to any one of claims 10-15, characterized in that, The first data packet also carries an end-of-retransmission flag, which indicates that the first data packet is the last data packet to undergo deduplication.

17. A data packet deduplication device, characterized in that, Included in the receiving device, the apparatus includes: The first receiving module is used to receive the first data packet; The deduplication module is used to perform deduplication processing on the first data packet according to the deduplication matching table if the first data packet carries a retransmission identifier, so that there is a first data packet in the receiving device. The deduplication matching table stores information about the data packets that the receiving device has received.

18. A data packet deduplication device, characterized in that, Included in the target forwarding device, the device includes: The sending module is configured to send a first data packet to the receiving device to instruct the receiving device to perform deduplication processing on the first data packet according to the deduplication matching table, so that there is a first data packet in the receiving device; wherein, the first data packet carries a retransmission identifier, and the deduplication matching table stores information on data packets that the receiving device has received.

19. A computer device, characterized in that, The computer device includes a processor and memory; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1-9, or to implement the steps of the method according to any one of claims 10-16.

20. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-9, or the steps of the method according to any one of claims 10-16.

21. A computer program product, characterized in that, The computer program product stores computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1-9, or the steps of the method according to any one of claims 10-16.

Images