Data packet type identification method and device, electronic equipment and storage medium

Abstract

The invention discloses a data packet type identification method and device, electronic equipment and a storage medium, and the method comprises the steps: receiving a first data packet, and determining a target difference value between a first serial number of the first data packet and a second serial number of a second data packet, and determining the data type of the first data packet according to the target difference value and the pre-stored corresponding relationship between the difference value of the serial numbers of two data packets which arrive in sequence and are adjacent and the type of the data packet which arrives later in the two data packets. By comparing the target difference value of the serial numbers of the first data packet and the second data packet, the data type of the corresponding first data packet is searched in the pre-stored corresponding relationship between the difference value of the serial numbers of two adjacent data packets arriving in sequence and the type of the data packet arriving later in the two data packets according to the target difference value. The related algorithm logic is simple, and complicated logic operation does not exist in the data processing process, so that the memory occupation is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a method and device for identifying a data packet type, an electronic device, and a storage medium. Background Art

[0002] In the process of data forwarding, when there are many types of data packets, such as a large amount of retransmitted data, multiple copies of the same data collected by multiple nodes, and replay attacks, the pressure on the back-end service or server to process and analyze data will increase. In the related technology, a complete set of TCP / IP protocol stacks is introduced, and the sliding window of the protocol stack is used to identify the data type of the data packet for corresponding processing. However, the architecture of the protocol stack is huge and the processing logic is complex. Using the protocol stack in some simple embedded systems is too wasteful of resources. For example, some simple data collection systems and data diversion systems will occupy more memory when using the protocol stack to identify the data type of the data packet. Summary of the invention

[0003] The present invention provides a method, device, electronic device and storage medium for data packet type identification. The main purpose is to provide a lightweight data packet type identification algorithm, thereby solving the problems of large-scale full-set protocol stack architecture introduced in the deduplication process, complex data packet data type identification logic, and large memory usage.

[0004] According to a first aspect of the present disclosure, a method for identifying a data packet type is provided, comprising:

[0005] receiving a first data packet;

[0006] Determine a target difference between a first sequence number of the first data packet and a second sequence number of a second data packet, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet;

[0007] The data type of the first data packet is determined according to the target difference and the pre-stored correspondence between the difference in sequence numbers of two adjacent data packets that arrive sequentially and the type of the data packet that arrives later in the two data packets.

[0008] Optionally, determining the data type of the first data packet according to the target difference and a pre-stored correspondence between a difference in sequence numbers of two adjacent data packets that arrive sequentially and are stored and a type of a data packet that arrives later of the two data packets comprises:

[0009] comparing the target difference with a preset difference;

[0010] If it is determined that the target difference is less than the preset difference, determining that the datagram type of the first data packet is a duplicate data packet type, and the duplicate data packet is a type that repeats the second sequence number of the second data packet;

[0011] If it is determined that the target difference is equal to the preset difference, then the data packet type of the first data packet is determined to be a sequential data packet type; the sequential data packet type is a type with a sequence number consecutive to the second sequence number of the second data packet;

[0012] If it is determined that the target difference is greater than the preset difference, the data packet type of the first data packet is determined to be an out-of-order data packet type, and the out-of-order data packet type is a type with a non-continuous sequence number with the second sequence number of the second data packet.

[0013] Optionally, before comparing the target difference with a preset difference, the method further includes:

[0014] determining a data packet length of the second data packet;

[0015] The data packet length is updated to the preset difference value.

[0016] Optionally, after determining that the datagram type of the first data packet is a repeated data packet type, the method includes:

[0017] Determine the sum of the second sequence number and the data packet length of the second data packet as the target expected sequence number;

[0018] Determine whether there is a third sequence number in the scrambled table that is the same as the target expected sequence number, and determine whether a data packet that is the same as the first data packet has been received before; wherein the scrambled table is used to store first record information of the scrambled data packet, and the first record information includes the sequence number and data length corresponding to the data packet of the scrambled data packet type.

[0019] Optionally, the method comprises:

[0020] In the case of determining that a data packet identical to the first data packet has been received before, receiving other data packets and determining data types of the other data packets, wherein the other data packets are data packets that arrive sequentially after the first data packet arrives and are adjacent to the first data packet;

[0021] When it is determined that a data packet identical to the first data packet has not been received before, the first sequence number and the data length of the first data packet are stored in the out-of-order table.

[0022] Optionally, after determining that the data packet type of the first data packet is an out-of-order data packet type, the method further includes:

[0023] Determine whether there is a fourth sequence number equal to the first sequence number in the random sequence table;

[0024] In the case where the fourth sequence number exists, determining that the data packet type of the first data packet is a random duplicate data packet type, and discarding the first data packet; the random duplicate data packet type is a type that repeats the sequence number of the random data packet in the random table;

[0025] When the fourth sequence number does not exist, the first sequence number and the data length of the first data packet are stored in the shuffle table.

[0026] Optionally, storing the first sequence number and the data length of the first data packet in the out-of-order table includes:

[0027] Determine whether there is a fifth sequence number in the random sequence table that is equal to the sum of the difference between the first sequence number and the preset value;

[0028] In the case where the fifth sequence number exists, the second record information corresponding to the first data packet and the third record information corresponding to the fourth data packet are merged into fourth record information, the fourth record information includes the first sequence number, the data length of the first data packet and the sum of the data length of the fourth data packet, the data packet corresponding to the fifth sequence number is the fourth data packet, and the second record information includes the data length and sequence number corresponding to the first data packet.

[0029] Storing the fourth record information in the random order table;

[0030] Delete the third record information corresponding to the fourth data packet in the out-of-sequence table.

[0031] According to a second aspect of the present disclosure, a device for identifying a data packet type is provided, comprising:

[0032] A receiving unit, configured to receive a first data packet;

[0033] A first determining unit, configured to determine a target difference between a first sequence number of the first data packet and a second sequence number of a second data packet, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet;

[0034] The second determination unit is used to determine the data type of the first data packet according to the target difference and the pre-saved correspondence between the difference in sequence numbers of two adjacent data packets that arrive sequentially and the type of the later-arriving data packet of the two data packets.

[0035] Optionally, the second determining unit includes:

[0036] A comparison module, used for comparing the target difference with a preset difference;

[0037] A first determining module, configured to determine that the datagram type of the first data packet is a repeated data packet type if it is determined that the target difference is less than a preset difference, and the repeated data packet is a type that repeats the second sequence number of the second data packet;

[0038] A second determination module, configured to determine that the data packet type of the first data packet is a sequential data packet type if it is determined that the target difference is equal to the preset difference; the sequential data packet type is a type with a sequence number consecutive to the second sequence number of the second data packet;

[0039] The third determination module is used to determine that the data packet type of the first data packet is a disordered data packet type if it is determined that the target difference is greater than the preset difference, and the disordered data packet type is a type with a non-continuous sequence number with the second sequence number of the second data packet.

[0040] Optionally, the second determining unit further includes:

[0041] a fourth determination module, in which the user determines the data packet length of the second data packet before comparing the target difference with a preset difference;

[0042] An updating module is used to update the length of the data packet to the preset difference.

[0043] Optionally, the second determining unit further includes:

[0044] a fifth determining module, configured to determine, after determining that the datagram type of the first data packet is a repeated data packet type, a sum of the second sequence number and a data packet length of the second data packet as a target expected sequence number;

[0045] The sixth determination module is used to determine whether there is a third sequence number in the out-of-order table that is identical to the target expected sequence number, and to determine whether a data packet identical to the first data packet has been received before; wherein the out-of-order table is used to store first record information of the out-of-order data packet, and the first record information includes the sequence number and data length corresponding to the data packet of the out-of-order data packet type.

[0046] Optionally, the sixth determining module is further used for:

[0047] In the case of determining that a data packet identical to the first data packet has been received before, receiving other data packets and determining data types of the other data packets, wherein the other data packets are data packets that arrive sequentially after the first data packet arrives and are adjacent to the first data packet;

[0048] When it is determined that a data packet identical to the first data packet has not been received before, the first sequence number and the data length of the first data packet are stored in the out-of-order table.

[0049] Optionally, the second determining unit further includes:

[0050] A judging module, configured to judge whether there is a fourth sequence number equal to the first sequence number in the out-of-sequence table after determining that the data packet type of the first data packet is an out-of-sequence data packet type;

[0051] a seventh determination module, configured to, in the presence of the fourth sequence number, determine that the data packet type of the first data packet is a random duplicate data packet type, and discard the first data packet; the random duplicate data packet type is a type having a sequence number that is repeated with that of the random data packet in the random table;

[0052] A storage module is used to store the first sequence number and the data length of the first data packet in the out-of-order table when the fourth sequence number does not exist.

[0053] Optionally, the storage module is further used for:

[0054] Determine whether there is a fifth sequence number in the random sequence table that is equal to the sum of the difference between the first sequence number and the preset value;

[0055] In the case where the fifth sequence number exists, the second record information corresponding to the first data packet and the third record information corresponding to the fourth data packet are merged into fourth record information, the fourth record information includes the first sequence number, the data length of the first data packet and the sum of the data length of the fourth data packet, the data packet corresponding to the fifth sequence number is the fourth data packet, and the second record information includes the data length and sequence number corresponding to the first data packet.

[0056] Storing the fourth record information in the random order table;

[0057] Delete the third record information corresponding to the fourth data packet in the out-of-sequence table.

[0058] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0059] at least one processor; and

[0060] a memory communicatively connected to the at least one processor; wherein,

[0061] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0062] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0063] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0064] The method, device, electronic device and storage medium for data packet type identification provided by the present disclosure mainly include: receiving a first data packet, determining a target difference between a first serial number of the first data packet and a second serial number of a second data packet, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet, and determining the data type of the first data packet according to the target difference and the correspondence between the difference between the serial numbers of two sequentially arrived and adjacent data packets and the type of the data packet that arrives later in the two data packets. Compared with the related art, by comparing the target difference between the serial numbers of the first data packet and the second data packet, according to the target difference, the corresponding data type of the first data packet is searched in the correspondence between the difference between the serial numbers of two sequentially arrived and adjacent data packets and the type of the data packet that arrives later in the two data packets. Among them, the algorithm logic involved is concise, so there will be no complicated logical operations in the process of processing the data, thereby saving memory usage.

[0065] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0067] Figure 1 A flowchart of a method for identifying a data packet type provided by an embodiment of the present disclosure;

[0068] Figure 2 A schematic diagram of the structure of a device for identifying data packet types provided in an embodiment of the present disclosure;

[0069] Figure 3A schematic diagram of another device for identifying data packet types provided in an embodiment of the present disclosure

[0070] Figure 4 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0071] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0072] The present disclosure provides a method, device, electronic device and storage medium for data packet type identification.

[0073] The following describes the method, device, electronic device and storage medium for data packet type identification according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0074] Figure 1 A flowchart of a method for identifying data packet types provided in an embodiment of the present disclosure.

[0075] like Figure 1 As shown, the method comprises the following steps:

[0076] Step 101: Receive a first data packet.

[0077] It should be understood that the above-mentioned first data packet refers generally to the data packets received during the data packet forwarding process, and the first data packet is only used for distinction and does not constitute a limitation of the present disclosure.

[0078] In the application scenario of data transmission, each data packet has a corresponding Transmission Control Protocol (TCP) header, and the TCP header contains information such as a sequence number and the data packet length of the corresponding data packet.

[0079] Step 102: determine a target difference between a first sequence number of the first data packet and a second sequence number of the second data packet.

[0080] After receiving the first data, a second data packet and a second sequence number of the second data packet are obtained, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet.

[0081] The first sequence number of the first data packet is obtained from the TCP header of the first data packet, the second sequence number of the second data packet is obtained from the TCP header of the second data packet, and the target difference between the first sequence number and the second sequence number is calculated. As an implementation method of an embodiment of the present application, the target difference can be a fixed size difference pre-set according to the transmission order of the data packets. As another implementation method provided by an embodiment of the present application, the target difference can also be the length of the first data packet, or the length of the second data packet, etc., which is not limited to the specific embodiment of the present application.

[0082] In practical applications, the target difference is used to ensure that data packets are transmitted in order, effectively achieving complete transmission of TCP data, and especially effectively correcting errors when errors occur in data transmission.

[0083] Step 103: Determine the data type of the first data packet according to the target difference and the pre-stored correspondence between the difference in sequence numbers of two adjacent data packets that arrive sequentially and the type of the later-arriving data packet of the two data packets.

[0084] The correspondence between the difference in serial numbers of two adjacent data packets that arrive sequentially and the type of the data packet that arrives later is pre-set, and the correspondence records different difference relationships, which are the correspondences between the duplicate data packet type, the sequential data packet type and the out-of-order data packet type, respectively.

[0085] In practical applications, after determining the target difference, the type of data packet can be determined by searching for the corresponding relationship. This implementation method does not require the use of complex logical calculations such as the protocol stack, but through simple calculations, the algorithm logic for searching the corresponding relationship is concise, thereby saving memory usage.

[0086] After determining that the data packet type is a duplicate data packet type, the duplicate data packet type is discarded to save memory in the data forwarding process. At the same time, the data in the forwarding process is streamlined to reduce the amount of data processing. This ensures that TCP data packets are transmitted in order, and can effectively achieve complete transmission of TCP data, especially when errors occur in data transmission, errors can be effectively corrected.

[0087] The method for identifying the type of data packet provided by the present disclosure receives a first data packet, determines a target difference between a first serial number of the first data packet and a second serial number of a second data packet, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet, and determines the data type of the first data packet according to the target difference and a pre-saved correspondence between the difference between the serial numbers of two sequentially arriving and adjacent data packets and the type of the data packet that arrives later in the two data packets. Compared with the related art, by comparing the target difference between the serial numbers of the first data packet and the second data packet, the data type of the corresponding first data packet is searched in the pre-saved correspondence between the difference between the serial numbers of two sequentially arriving and adjacent data packets and the type of the data packet that arrives later in the two data packets according to the target difference. The algorithm logic involved is concise, so there will be no complicated logical operations in the data processing process, thereby saving memory usage.

[0088] As a refinement of the above embodiment, in step 103, when determining the data type of the first data packet according to the target difference and the pre-stored correspondence between the difference in sequence numbers of two sequentially arriving and adjacent data packets and the type of the data packet that arrives later in the two data packets, the following implementation methods may be adopted but are not limited to: comparing the target difference with the preset difference; and determining the following three types of data packets according to the comparison result:

[0089] Type 1: Repeated packet type

[0090] If it is determined that the target difference is less than the preset difference, determining that the datagram type of the first data packet is a duplicate data packet type, and the duplicate data packet is a type that repeats the second sequence number of the second data packet;

[0091] Type 2: Sequential data packet type

[0092] If it is determined that the target difference is equal to the preset difference, the data packet type of the first data packet is determined to be a sequential data packet type; the sequential data packet type is a type with a sequence number continuous with the second sequence number of the second data packet.

[0093] Type 3: Out-of-order data packet type

[0094] If it is determined that the target difference is greater than the preset difference, the data packet type of the first data packet is determined to be an out-of-order data packet type, and the out-of-order data packet type is a type with a non-continuous sequence number with the second sequence number of the second data packet.

[0095] As an implementable manner of the embodiment of the present application, before comparing the target difference with the preset difference, the packet length of the second data packet is determined, and the packet length is updated to the preset difference. The packet length of the second data packet can be obtained from the TCP header of the second data packet. In this case, the target difference can also be the length of the second data packet.

[0096] After determining that the packet type of the first data packet is a duplicate data packet type, the sum of the second sequence number and the packet length of the second data packet is determined as the target expected sequence number, and it is determined whether there is a third sequence number that is the same as the target expected sequence number in the scrambled table, and whether a data packet that is the same as the first data packet has been received before; wherein the scrambled table is used to store first record information of the scrambled data packet, and the first record information includes the sequence number and data length corresponding to the data packet of the scrambled data packet type.

[0097] The target expected sequence number is determined by using the sum of the second sequence number and the data length of the second data packet, the purpose of which is to obtain the sequence number of the next data packet (i.e., the first sequence number of the first data packet), that is, the target expected sequence number corresponding to the next data packet (the first data packet), which is obtained by adding the second sequence number and the data length of the second data packet. It can be understood that the preset difference is the data length of the second data packet.

[0098] In an embodiment of the present application, a scrambled table is used to store first record information of scrambled data packets. Using the scrambled table to store the record information of the scrambled data packets can reduce the loss of transmitted data. At the same time, the record information of the data packets existing in the scrambled table is judged in real time, and the data packet information existing in the scrambled table is cleaned up in time. The judgment of whether there is a third sequence number in the scrambled table that is the same as the target expected sequence number is to judge whether there is a data packet in the scrambled table that is after the first data packet and continuous with the first data packet.

[0099] If it is determined that the data packet identical to the first data packet has not been received before, the first sequence number and the data length of the first data packet are stored in the disorder table. If there is no third sequence number and it is determined that the data packet identical to the first data packet has been received before, other data packets are received and the data types of the other data packets are determined. The other data packets are data packets that arrive sequentially after the first data packet arrives and are adjacent to the first data packet.

[0100] As an extension of the above embodiment, after determining that the data packet type of the first data packet is a disordered data packet type, the method may also adopt but is not limited to the following implementation manner: determining whether there is a fourth sequence number equal to the first sequence number in the disordered table; if the fourth sequence number exists, determining that the first data packet is a disordered duplicate data packet type, which is a type that repeats the sequence number of the disordered data packet in the disordered table, and discarding the first data packet; if the fourth sequence number does not exist, storing the first sequence number and the data length of the first data packet in the disordered table, and forwarding the first data packet corresponding to the first sequence number.

[0101] As a refinement of the above embodiment, it is determined that the first data packet is the out-of-order data packet, that is, the data packet received in advance during the forwarding process. At this time, the first sequence number is also used to determine whether the first data packet is an out-of-order data packet. In order to determine whether the data packet information in the out-of-order table is repeated with the information of the first data packet, that is, to determine whether the fourth sequence number that is the same as the first sequence number exists in the out-of-order table. When the fourth sequence number that is the same as the first sequence number exists in the out-of-order table, it indicates that the information of the data packet that is repeated with the first data packet already exists in the out-of-order table. At this time, the first data packet is discarded; when the fourth sequence number does not exist in the out-of-order table, it indicates that the information of the data packet that is repeated with the first data packet does not exist in the out-of-order table. At this time, the information of the first data packet, that is, the first sequence number and the data length of the first data packet are stored in the out-of-order table. By storing the information of the out-of-order data packet in the out-of-order table, the out-of-order data packet is deduplicated and the first data packet is forwarded.

[0102] As a refinement of the above embodiment, after storing the first sequence number and the data length of the first data packet in the shuffle table, determining whether there is a fifth sequence number in the shuffle table that is equal to the sum of the first sequence number and the data length of the first data packet;

[0103] In order to determine whether there is a fifth sequence number in the shuffle table that is equal to the sum of the difference between the first sequence number and the preset value, that is, to determine whether there is a fifth sequence number in the shuffle table that is equal to the sum of the first sequence number and the data length of the second data packet. In the case where the fifth sequence number exists, the third record information corresponding to the first data packet and the third record information corresponding to the fourth data packet are combined into fourth record information, the fourth record information includes the first sequence number, the data length of the first data packet and the sum of the data length of the fourth data packet, the data packet corresponding to the fifth sequence number is the fourth data packet, and the second record information includes the data length and sequence number corresponding to the first data packet.

[0104] When the fifth sequence number exists in the scrambled table, the second record information corresponding to the first sequence number and the third record information corresponding to the fourth data packet are merged into fourth record information, the fourth record information includes the first sequence number, the data length of the first data packet and the sum of the data length of the fourth data packet, the data packet corresponding to the fifth sequence number is the fourth data packet, and the second record information includes the data length and sequence number corresponding to the first data packet.

[0105] The fourth record information is stored in the random order table, and the third record information corresponding to the fourth data packet in the random order table is deleted to reduce the memory occupation of the random order table.

[0106] As a refinement of the above embodiment, before determining the data type of the first data packet according to the target difference and the pre-saved correspondence between the difference between the sequence numbers of two sequentially arrived and adjacent data packets and the type of the data packet that arrives later in the two data packets, the method can also adopt but is not limited to the following implementation methods, for example: judging whether the data length of the first data packet is zero; if it is zero, and the first data packet is a synchronous data packet, determining the value of the target expected sequence number to be zero. If the data length of the first data packet is not zero, it means that the first data packet carries data, so that step 103 is executed to determine the data type of the first data packet according to the target difference and the pre-saved correspondence between the difference between the sequence numbers of two sequentially arrived and adjacent data packets and the type of the data packet that arrives later in the two data packets; if the data length of the first data packet is zero, it means that the first data packet does not carry data, and when the first data packet is a synchronous data packet, determining the initial value of the preset expected sequence number to be zero. If the first data packet is not a synchronous data packet, but only an ordinary data packet that does not carry data that appears during the data forwarding process, the forwarded ordinary data packet is not processed at this time. The above description is only an exemplary description and does not constitute a limitation of the present disclosure.

[0107] As a refinement of the above embodiment, the first data packet includes a first synchronization data packet. After determining that the value of the target expected sequence number is zero, the method may also adopt but is not limited to the following implementation methods, for example: when the first synchronization data packet is sent on the first side of the session, the value of the target expected sequence number of the first side session is the sequence number generated by the first side session plus one; when the second synchronization data packet and the confirmation data packet are sent on the second side of the session, the target expected sequence number value of the second side session is the sequence number generated by the second side session plus one.

[0108] As a refinement of the above embodiment, when a session performs a three-way handshake, in the case where the first side of the session sends a first synchronization data packet, the target expected sequence number value of the session on the first side is the sequence number generated by the session on the first side plus one; in the case where the second side of the session sends a second synchronization data packet and a confirmation data packet, the target expected sequence number value of the session on the second side is the sequence number generated by the session on the second side plus one. That is, the values ​​of the target expected sequence numbers on different sides of the session are generated based on the sequence number generated by the session on this side, and this is also the case in actual applications of the present disclosure.

[0109] As a refinement of the above embodiment, before determining whether the data length of the first data packet is zero, the method may also adopt but is not limited to the following implementation methods, for example: querying the corresponding table entry in the flow table according to the five-tuple contained in the first data packet, the five-tuple including the source address, the destination address, the source port, the destination port, and the protocol type; under the corresponding table entry, determining whether the data length of the first data packet is zero.

[0110] That is, the corresponding table entry is queried in the flow table through the five-tuple data carried in the first data packet. After determining the corresponding table entry, it is judged under the table entry whether the data length of the first data packet is zero. It should be understood that the flow table is equivalent to a collection of different forwarding strategies, and one table entry corresponds to one forwarding strategy. Therefore, it is necessary to match the corresponding table entry for the first data packet in advance.

[0111] The main content disclosed in the present invention is the method for determining duplications and storing as few sequence numbers as possible, which can be briefly summarized as follows: when a data packet passes, the expected sequence number of the next packet is calculated based on the sequence number and data length of the current data packet. If it is before the data packet is received, it is considered to be a duplicate, otherwise it is considered to be sequential or disordered. Inevitably, disordered packets will appear in network transmission. At this time, two sets of key values ​​need to be recorded: the next expected continuous sequence number that has arrived in sequence and all disordered sequence numbers and data lengths. If the continuity conditions are met between all disordered sequence numbers and data lengths, they can be merged to form a continuous larger interval. After merging, several discontinuous intervals are formed as a whole to reduce storage.

[0112] The table entry data structure involved in the present disclosure needs to record two fields of information: the first field stores the target expected sequence number, that is, the expected next maximum continuous sequence number; the second field stores the disordered table, pointing to a temporarily disordered sequence number set, and its implementation method includes but is not limited to a hash table, a linked list, an array, etc., which stores the combined sequence number and data length of all disordered data packets.

[0113] In order to more clearly illustrate the steps of the embodiment of the present disclosure, the following is a scenario description provided by the present disclosure: For the same session, there will be an initial sequence number when the session is just connected, and the value of the target expected sequence number is initialized to the value of the initial sequence number plus n. When a data packet passes, if the sequence number of the data packet is after the target expected sequence number, the data packet is considered to be a disordered data packet; if the two are equal, the target expected sequence number of the next packet is calculated according to the sequence number of the data packet and the data length of the data packet, and the target expected sequence number value is updated. If the next data packet is before the updated target expected sequence number, it is considered to be a duplicate, and the data packet after it is considered to be a disordered data packet, and so on. The sequence number of the disordered packet is temporarily stored in the disordered table. The target expected sequence number calculated each time is first compared with the disordered table. If there is an expected value in the disordered table, the record is deleted from the disordered table, and the next target expected sequence number is calculated, and the disordered table is still compared again until it cannot be compared (there is no longer an expected value or the disordered table is empty), and the current final target expected sequence number is updated once. In particular, when the sequence number of an out-of-order data packet is stored in the out-of-order table, it is possible to first determine whether the sum of the sequence number and its data length exists in the out-of-order table. If so, they can be merged into one record, which includes the sequence number and the sum of the data lengths of the two records. The other record can be deleted to reduce storage. Similarly, other out-of-order records can be compared repeatedly until they cannot be merged.

[0114] In order to more clearly show the processing flow of related data packets when deduplication of data packets is performed, please refer to Table 1, where seq is the sequence number of the data packet, expect seq is the target expected sequence number, and also includes a disorder table and optional steps. In order to more clearly show the processing flow of related data packets, the data length of each data packet is assumed to be 1, and the following Table 1 can be constructed:

[0115] Table 1

[0116]

[0117] In the present invention, the storage of the disordered table can be a hash table, a linked list, or an array. In the present embodiment, a hash table is preferably used. The following is a comparison result of the memory size occupied by using a hash table to store the data packet information and using a bit map to store the data packet information, as shown below:

[0118] Comparison with bitmap memory usage (B: byte bit: bit)

[0119] If a TCP connection wants to send 1GB of data (assuming an average message payload of 512B),

[0120] Bitmap storage:

[0121] 1GB = 1024*1024*1024B, one byte is represented by one bit. 1024*1024*1024 bits are needed.

[0122] Memory occupied = (1024*1024*1024bit) / (8bit / B) = 134,217,728B (128MB)

[0123] Hash table storage:

[0124] In the worst case, using a hash linked list for storage requires storing all packet sequence numbers and data length entries in the linked list. Assuming that this entry is stored in a 64-bit long integer (8B) (this is already the limit and is generally not reached), and the pointer needs to use a 64-bit long integer (8B), then one record entry occupies 16B.

[0125] Memory occupied = (1024*1024*1024B) / 512B*(16B) = 33,554,432B (32MB)

[0126] It can be seen that the storage space occupied by the hash table is one-fourth of that of the bitmap. Moreover, since the entries in the hash table can be merged, it is impossible to reach the worst case, and the memory usage is much better than that of the bitmap.

[0127] In summary, this embodiment can achieve the following effects:

[0128] 1. Determine the sequence number of the data packet to be forwarded based on the target expected sequence number. When the sequence number of the received data packet is less than the target expected sequence number, it means that the data packet has been forwarded, so the data packet is a duplicate data packet, and the duplicate data packet is discarded. The algorithm logic involved is simple, so there will be no complicated logical operations in the data processing process, thereby saving memory usage.

[0129] 2. This embodiment is relatively independent and has low coupling with other modules, avoiding the problem of introducing a full set of protocol stacks that are too large, have low performance, and occupy a lot of memory. The algorithm of this embodiment is simple in logic, solving the problem of complex logic and complex implementation of similar sliding windows. Because this embodiment has low coupling with other modules and strong applicability, it can not only adapt to various other systems that require deduplication, but also can be used in different process stages of the system.

[0130] 3. The algorithm of this embodiment is simple to implement, requiring only addition operations, comparison operations, very few loops, and some necessary table lookup and comparison operations, so the performance is very high and the impact on the overall system performance is very small.

[0131] 4. This embodiment has fewer record fields and has the function of merging continuous records, so it occupies less memory. Moreover, the memory occupancy in the best case, the worst case, and the average case is lower than that in other implementations.

[0132] 5. You only need to add a set of hash tables to implement a flow table consisting of five tuples, which is convenient for expansion.

[0133] Corresponding to the above-mentioned method for identifying data packet types, the present invention also provides a device for identifying data packet types. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0134] Figure 2 A schematic diagram of a device for identifying a data packet type provided in an embodiment of the present disclosure is shown in FIG. Figure 2 As shown, including:

[0135] The receiving unit 21 is configured to receive a first data packet;

[0136] A first determining unit 22 is used to determine a target difference between a first sequence number of the first data packet and a second sequence number of a second data packet, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet;

[0137] The second determination unit 23 is used to determine the data type of the first data packet according to the target difference and the pre-stored correspondence between the difference in sequence numbers of two adjacent data packets that arrive sequentially and the type of the later-arriving data packet of the two data packets.

[0138] The device for identifying the type of data packet provided by the present disclosure has a main technical solution including: receiving a first data packet, determining a target difference between a first serial number of the first data packet and a second serial number of a second data packet, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet, and determining the data type of the first data packet according to the target difference and the correspondence between the difference between the serial numbers of two sequentially arrived and adjacent data packets and the type of the data packet that arrives later in the two data packets. Compared with the related art, by comparing the target difference between the serial numbers of the first data packet and the second data packet, the data type of the corresponding first data packet is searched in the correspondence between the difference between the serial numbers of two sequentially arrived and adjacent data packets and the type of the data packet that arrives later in the two data packets according to the target difference. The algorithm logic involved is concise, so there will be no complicated logical operations in the process of processing the data, thereby saving memory usage.

[0139] Furthermore, in a possible implementation of this embodiment, as Figure 3 As shown, the second determining unit 23 includes:

[0140] A comparison module 231, configured to compare the target difference with a preset difference;

[0141] A first determining module 232, configured to determine that the datagram type of the first data packet is a repeated data packet type if it is determined that the target difference is less than a preset difference, and the repeated data packet is a type that repeats the second sequence number of the second data packet;

[0142] The second determination module 233 is used to determine that the data packet type of the first data packet is a sequential data packet type if it is determined that the target difference is equal to the preset difference; the sequential data packet type is a type with a sequence number consecutive to the second sequence number of the second data packet;

[0143] The third determination module 234 is used to determine that the data packet type of the first data packet is a disordered data packet type if it is determined that the target difference is greater than the preset difference, and the disordered data packet type is a type with a non-continuous sequence number with the second sequence number of the second data packet.

[0144] Furthermore, in a possible implementation of this embodiment, as Figure 3 As shown, the second determining unit 23 further includes:

[0145] A fourth determination module 235, in which the user determines the data packet length of the second data packet before comparing the target difference with a preset difference;

[0146] The updating module 236 is used to update the data packet length to the preset difference value.

[0147] Furthermore, in a possible implementation of this embodiment, as Figure 3 As shown, the second determining unit also includes:

[0148] a fifth determining module 237, configured to determine, after determining that the datagram type of the first data packet is a repeated data packet type, a sum of the second sequence number and a data packet length of the second data packet as a target expected sequence number;

[0149] The sixth determination module 238 is used to determine whether there is a third sequence number in the out-of-order table that is identical to the target expected sequence number, and to determine whether a data packet identical to the first data packet has been received before; wherein the out-of-order table is used to store first record information of the out-of-order data packet, and the first record information includes the sequence number and data length corresponding to the data packet of the out-of-order data packet type.

[0150] Furthermore, in a possible implementation of this embodiment, as Figure 3 As shown, the sixth determination module 238 is further used for:

[0151] In the case of determining that a data packet identical to the first data packet has been received before, receiving other data packets and determining data types of the other data packets, wherein the other data packets are data packets that arrive sequentially after the first data packet arrives and are adjacent to the first data packet;

[0152] When it is determined that a data packet identical to the first data packet has not been received before, the first sequence number and the data length of the first data packet are stored in the out-of-order table.

[0153] Furthermore, in a possible implementation of this embodiment, as Figure 3 As shown, the second determining unit also includes:

[0154] A judging module 239, configured to judge whether there is a fourth sequence number equal to the first sequence number in the out-of-sequence table after determining that the data packet type of the first data packet is an out-of-sequence data packet type;

[0155] a seventh determination module 2310, configured to, in the case where the fourth sequence number exists, determine that the data packet type of the first data packet is a random duplicate data packet type, and discard the first data packet; the random duplicate data packet type is a type having a sequence number that is repeated with that of the random data packet in the random table;

[0156] The storage module 2311 is used to store the first sequence number and the data length of the first data packet in the out-of-order table when the fourth sequence number does not exist.

[0157] Furthermore, in a possible implementation of this embodiment, as Figure 3 As shown, the storage module 2311 is also used for:

[0158] Determine whether there is a fifth sequence number in the random sequence table that is equal to the sum of the difference between the first sequence number and the preset value;

[0159] In the case where the fifth sequence number exists, the second record information corresponding to the first data packet and the third record information corresponding to the fourth data packet are merged into fourth record information, the fourth record information includes the first sequence number, the data length of the first data packet and the sum of the data length of the fourth data packet, the data packet corresponding to the fifth sequence number is the fourth data packet, and the second record information includes the data length and sequence number corresponding to the first data packet.

[0160] Storing the fourth record information in the random order table;

[0161] Delete the third record information corresponding to the fourth data packet in the out-of-sequence table.

[0162] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0163] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0164] Figure 4 A schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0165] like Figure 4 As shown, the device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 302 or a computer program loaded from a storage unit 308 to a RAM (Random Access Memory) 303. In the RAM 303, various programs and data required for the operation of the device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An I / O (Input / Output) interface 305 is also connected to the bus 304.

[0166] A number of components in the device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0167] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as a method for packet type identification. For example, in some embodiments, the method for packet type identification may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform the aforementioned method for identifying the type of data packet by any other appropriate means (eg, by means of firmware).

[0168] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0169] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0170] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory) or a flash memory, an optical fiber, a CD-ROM (Compact Dis sc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0171] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0172] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0173] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0174] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0175] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0176] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for identifying a data packet type, characterized in that: include: receiving a first data packet; Determine a target difference between a first sequence number of the first data packet and a second sequence number of a second data packet, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet; The data type of the first data packet is determined according to the target difference and the pre-stored correspondence between the difference in sequence numbers of two adjacent data packets that arrive sequentially and the type of the data packet that arrives later in the two data packets.

2. The method according to claim 1, characterized in that Determining the data type of the first data packet according to the target difference and the pre-stored correspondence between the difference in sequence numbers of two sequentially arriving and adjacent data packets and the type of the data packet that arrives later in the two data packets includes: comparing the target difference with a preset difference; If it is determined that the target difference is less than the preset difference, determining that the datagram type of the first data packet is a duplicate data packet type, and the duplicate data packet is a type that repeats the second sequence number of the second data packet; If it is determined that the target difference is equal to the preset difference, then the data packet type of the first data packet is determined to be a sequential data packet type; the sequential data packet type is a type with a sequence number consecutive to the second sequence number of the second data packet; If it is determined that the target difference is greater than the preset difference, the data packet type of the first data packet is determined to be an out-of-order data packet type, and the out-of-order data packet type is a type with a non-continuous sequence number with the second sequence number of the second data packet.

3. The method according to claim 2, characterized in that Before comparing the target difference with the preset difference, the method further includes: determining a data packet length of the second data packet; The data packet length is updated to the preset difference value.

4. The method according to claim 2, characterized in that: After determining that the datagram type of the first data packet is a repeated data packet type, the method includes: Determine the sum of the second sequence number and the data packet length of the second data packet as the target expected sequence number; Determine whether there is a third sequence number in the scrambled table that is the same as the target expected sequence number, and determine whether a data packet that is the same as the first data packet has been received before; wherein the scrambled table is used to store first record information of the scrambled data packet, and the first record information includes the sequence number and data length corresponding to the data packet of the scrambled data packet type.

5. The method according to claim 4, characterized in that The method comprises: In the case of determining that a data packet identical to the first data packet has been received before, receiving other data packets and determining data types of the other data packets, wherein the other data packets are data packets that arrive sequentially after the first data packet arrives and are adjacent to the first data packet; When it is determined that a data packet identical to the first data packet has not been received before, the first sequence number and the data length of the first data packet are stored in the out-of-order table.

6. The method according to claim 5, characterized in that After determining that the data packet type of the first data packet is an out-of-order data packet type, the method further includes: Determine whether there is a fourth sequence number equal to the first sequence number in the random sequence table; In the case where the fourth sequence number exists, determining that the data packet type of the first data packet is a random duplicate data packet type, and discarding the first data packet; the random duplicate data packet type is a type that repeats the sequence number of the random data packet in the random table; When the fourth sequence number does not exist, the first sequence number and the data length of the first data packet are stored in the shuffle table.

7. The method according to claim 6, characterized in that The storing the first sequence number and the data length of the first data packet in the out-of-order table comprises: Determine whether there is a fifth sequence number in the random sequence table that is equal to the sum of the difference between the first sequence number and the preset value; In the case where the fifth sequence number exists, the second record information corresponding to the first data packet and the third record information corresponding to the fourth data packet are combined into fourth record information, the fourth record information including the first sequence number, the sum of the data length of the first data packet and the data length of the fourth data packet, the data packet corresponding to the fifth sequence number is the fourth data packet, and the second record information includes the data length and sequence number corresponding to the first data packet; Storing the fourth record information in the random order table; Delete the third record information corresponding to the fourth data packet in the out-of-sequence table.

8. A device for identifying data packet type, characterized in that: include: A receiving unit, configured to receive a first data packet; A first determining unit, configured to determine a target difference between a first sequence number of the first data packet and a second sequence number of a second data packet, wherein the second data packet is a data packet that arrives sequentially before the first data packet arrives and is adjacent to the first data packet; The second determination unit is used to determine the data type of the first data packet according to the target difference and the pre-saved correspondence between the difference in sequence numbers of two adjacent data packets that arrive sequentially and the type of the later-arriving data packet of the two data packets.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

11. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images