Method and apparatus for accelerating data transmission based on maximum transmission unit

Abstract

The embodiment of the application discloses a data transmission acceleration method and device based on a maximum transmission unit, wherein the method comprises the following steps: obtaining the MTU of a next device, determining the size of the fillable space of a current data packet according to the MTU and the space size of the existing data of the current data packet; obtaining a target prediction dictionary corresponding to the size of the fillable space; obtaining the data of the fillable space according to the existing data of the current data packet and the target prediction dictionary, wherein the similarity between the data of the fillable space and the preceding data of a next data packet is greater than a preset threshold; generating a filled current data packet according to the existing data of the current data packet and the data of the fillable space, and sending the filled current data packet. The embodiment of the application determines the fillable space size of the current data packet based on the MTU, and obtains the data of the fillable space in combination with the target prediction dictionary, so that the device generates a data packet carrying more data, and the effect of improving the data transmission efficiency is achieved.

Description

Technical Field

[0001] This application relates to the field of data encoding, and in particular to a data transmission acceleration method and apparatus based on the maximum transmission unit. Background Technology

[0002] The maximum transmission unit (MTU) is the largest frame or data packet that can be transmitted over a data link—measured in bytes or octets. It is most commonly used as a reference for packet size on Ethernet networks using the Internet Protocol (IP).

[0003] Each device in a network has a Maximum Transmission Unit (MTU) that it can receive and transmit. The MTU of the next receiving device is determined by the previous device before it sends a data packet. If a data packet is too large for the next receiving device to accept, the previous device will split the packet into multiple packets for transmission; this is called fragmentation.

[0004] Fragmentation increases latency and adds extra data, negatively impacting network transmission performance. For optimal performance, the MTU of the original sending device is set as large as possible, but still smaller than the MTU of all devices in the network between the sender and the final receiver. In practice, the sending device doesn't know the MTU of all intermediate devices, but only the MTU of the next device in the queue.

[0005] The efficiency of data transmission in a network can be maximized by utilizing the MTU (Mean Transmission Unit) within the network, thereby accelerating data transmission speed. However, obtaining the data within the MTU and generating the MTU itself is a challenge. Summary of the Invention

[0006] This application provides a data transmission acceleration method and apparatus based on the maximum transmission unit (MTU). By obtaining the size of the fillable space in the MTU of the current data packet and then obtaining the target prediction dictionary to predict the data in the fillable space, the device generates and sends data packets, thereby improving data transmission efficiency.

[0007] Firstly, a data transmission acceleration method based on the maximum transmission unit is provided, the method comprising:

[0008] Obtain the MTU of the next device, and determine the size of the padding space for the current data packet based on the MTU and the space size of the existing data in the current data packet; obtain the target prediction dictionary corresponding to the size of the padding space, where the target prediction dictionary is one of multiple prediction dictionaries; obtain the data for the padding space based on the existing data of the current data packet and the target prediction dictionary, where the similarity between the data for the padding space and the previous data of the next data packet is greater than a preset threshold; generate the current data packet based on the existing data of the current data packet and the data for the padding space, and send the current data packet.

[0009] As can be seen, in this embodiment, the size of the fillable space in the MTU is determined by obtaining the MTU used for data transmission. Then, a target prediction dictionary is obtained, and the data for the fillable space is predicted based on the target prediction dictionary. Finally, the current data packet is generated based on the existing data and the data for the fillable space, and then sent. Since the current data packet is generated based on the MTU size, this ultimately improves data transmission efficiency and speeds up data transmission.

[0010] In one possible example, the size of the padding space for the current packet is determined based on the MTU and the space size of the existing data in the current packet, including: determining the payload of the current packet based on the MTU; and determining the size of the padding space for the current packet based on the difference between the payload of the current packet and the space size of the existing data.

[0011] In one possible example, the data to fill the space is obtained based on the existing data of the current data packet and the target prediction dictionary, including: matching the existing data of the current data packet with the target prediction dictionary to obtain the prediction items corresponding to the existing data, wherein the target prediction dictionary includes multiple prediction items and the size of the prediction items corresponds to the size of the fillable space; inputting the prediction items into the data prediction model, and obtaining the data to fill the space based on the prediction results of the prediction model.

[0012] In one possible example, before obtaining the target prediction dictionary corresponding to the fillable space, the method further includes: generating multiple initial prediction dictionaries, each prediction dictionary corresponding to a possible space size, and each prediction dictionary including multiple initial prediction items; processing each of the multiple initial prediction dictionaries to remove invalid initial prediction items from the multiple initial prediction items in each initial prediction dictionary, thereby obtaining multiple prediction dictionaries.

[0013] As can be seen, in this embodiment of the application, after generating the corresponding initial prediction dictionary according to the data space size, it is simplified by removing invalid initial prediction items. The resulting prediction dictionary can greatly reduce its space occupation, reduce the power consumption of the device in storing and reading the prediction dictionary, and thus improve the efficiency of using the prediction dictionary for data prediction.

[0014] In one possible example, an invalid initial prediction satisfies at least one of the following: a duplicate prediction, a prediction without semantic meaning, or a prediction with incorrect semantic meaning.

[0015] In one possible example, generating a padded current data packet based on the existing data and the data of the padded space in the current data packet includes: appending the data of the padded space to the existing data in the current data packet to generate the padded current data packet.

[0016] In one possible example, the MTU also includes middleware adding data and sending the padded current data packet, including: sending the padded current data packet and sending indication information, which is used to indicate the size of the space of the existing data in the padded current data packet, or also to indicate the size of the padded space in the padded current data packet;

[0017] The method further includes: receiving a response message, which indicates the size of the space of existing data in the current data packet after padding received by the next device, or further indicates the size of the padding space in the current data packet after padding.

[0018] As can be seen, in this embodiment, by sending an indication message to the next device to indicate the size of the existing data space in the current data packet after filling, it is possible to ensure that truly important data, that is, the existing data in the current data packet, can be completely sent to the next device. Other non-important data, including data that can be filled and data added by middleware, are exempt from the confirmation process, thus improving data transmission efficiency. If the indication message also indicates the size of the fillable space in the current data packet after filling, it is possible to confirm whether the data in the fillable space has been completely sent to the next device. This ensures that the transmission of the current data packet after filling is lossless, preventing the data packet from being corrupted at the next device.

[0019] Secondly, a data transmission apparatus is provided, the apparatus comprising:

[0020] The acquisition unit is used to acquire the MTU of the next device and determine the size of the fillable space of the current data packet based on the MTU and the space size of the existing data in the current data packet.

[0021] The acquisition unit is also used to acquire a target prediction dictionary corresponding to the size of the fillable space, wherein the target prediction dictionary is one of multiple prediction dictionaries;

[0022] The prediction unit is used to obtain data that can fill the space based on the existing data of the current data packet and the target prediction dictionary. The data that can fill the space has a similarity to the previous data of the next data packet that is greater than a preset threshold.

[0023] The transceiver unit is used to generate a padded current data packet based on the existing data and the available space in the current data packet, and then send the padded current data packet.

[0024] In one possible example, the acquisition unit is specifically used to: determine the payload of the current data packet based on the MTU; and determine the size of the fillable space for the current data packet based on the difference between the payload of the current data packet and the space size of existing data.

[0025] In one possible example, the prediction unit is specifically used to: match the existing data of the current data packet with the target prediction dictionary to obtain the prediction item corresponding to the existing data, wherein the target prediction dictionary includes multiple prediction items and the size of the prediction item corresponds to the size of the fillable space; input the prediction item into the data prediction model, and obtain the data of the fillable space based on the prediction result of the prediction model.

[0026] In one possible example, before obtaining the target prediction dictionary corresponding to the fillable space, the obtaining unit is further configured to: generate multiple initial prediction dictionaries, each prediction dictionary corresponding to a possible space size, and each prediction dictionary including multiple initial prediction items; process each of the multiple initial prediction dictionaries to remove invalid initial prediction items from the multiple initial prediction items in each initial prediction dictionary, thereby obtaining multiple prediction dictionaries.

[0027] In one possible example, an invalid initial prediction satisfies at least one of the following: a duplicate prediction, a prediction without semantic meaning, or a prediction with incorrect semantic meaning.

[0028] In one possible example, the transceiver unit is specifically used to: append data with fillable space to the existing data in the current data packet to generate the padded current data packet.

[0029] In one possible example, the MTU also includes middleware to add data. The transceiver unit is specifically used to: send the padded current data packet and send indication information, which is used to indicate the size of the space of the existing data in the padded current data packet, or also to indicate the size of the padded space in the padded current data packet.

[0030] The transceiver unit is also used to: receive a response message, which indicates the size of the existing data in the current data packet after padding received by the next device, or it is also used to indicate the size of the padding space in the current data packet after padding.

[0031] Thirdly, an electronic device is provided, comprising: a processor, a memory, and a communication interface, wherein the processor, the memory, and the communication interface are interconnected and perform communication between them;

[0032] The memory stores executable program code, and the communication interface is used for wireless communication.

[0033] The processor is used to retrieve executable program code stored in memory and execute some or all of the steps described in the first aspect of the embodiments of this application or any method of the first aspect.

[0034] Fourthly, embodiments of this application provide a chip system, including: a processor coupled to a memory, the memory being used to store programs or instructions, wherein when the program or instructions are executed by the processor, the chip system implements the method described in the first aspect or any one of the first aspects.

[0035] Optionally, the chip system also includes an interface circuit for exchanging computer instructions with the processor.

[0036] Optionally, the chip system may include one or more processors, which can be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor that reads software code stored in memory.

[0037] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application does not specifically limit the type of memory or the arrangement of the memory and processor.

[0038] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed, causes a computer to perform the method described in the first aspect or any one of the first aspects.

[0039] Sixthly, embodiments of this application provide a computer program product, wherein the computer program product includes a computer program operable to cause a computer to perform some or all of the steps described in the first aspect or any method of the first aspect of the embodiments of this application. The computer program product may be a software installation package.

[0040] In a seventh aspect, embodiments of this application provide a data transmission system, including the data transmission apparatus described in the second aspect. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1A This is a schematic diagram of the network architecture scenario to which the embodiments of this application apply;

[0043] Figure 1B This application provides a schematic diagram of a protocol encapsulation process in a network model.

[0044] Figure 1C This is a schematic diagram of the structure of an Ethernet frame provided in an embodiment of this application;

[0045] Figure 2A A flowchart of a data transmission acceleration method based on the maximum transmission unit provided in this application embodiment;

[0046] Figure 2B This is a schematic diagram of multi-data packet transmission provided in an embodiment of this application;

[0047] Figure 2C A schematic diagram of a prediction dictionary provided for an embodiment of this application;

[0048] Figure 2D A schematic diagram of a prediction process provided in an embodiment of this application;

[0049] Figure 3 This is a schematic diagram of the structure of a data transmission device provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0052] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps is not limited to the steps listed, but may optionally include steps not listed, or may optionally include other steps inherent to these processes, methods, products, or apparatuses.

[0053] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0054] First, please refer to Figure 1A , Figure 1A This is a schematic diagram of the network architecture scenario applicable to the embodiments of this application, such as... Figure 1A As shown, this network architecture includes a current device and a next device. The current device and the next device are the devices that perform network data transmission. Specifically, they can be a terminal device and a network device, a terminal device and a terminal device, or a network device and a network device. Alternatively, this network architecture may also include at least one middleware. The middleware can be a router, a virtual private network (VPN) server, or a gateway, etc. The current device is used to send data packets to the next device. When middleware is included, the data packets sent by the current device reach the next device through the middleware.

[0055] Then, please see Figure 1B , Figure 1B This application provides a schematic diagram of a protocol encapsulation process in a network model, as shown in the embodiments. Figure 1B As shown, user data is encapsulated using the TCP protocol at the transport layer, the IP protocol at the network layer, and the Ethernet protocol at the link layer. The encapsulation process includes adding a header before the data packet.

[0056] MTU applies to both the data link layer and the network layer. MTU parameters are typically associated with the communication interface (network interface card, NIC, or serial port, etc.). Standards (such as Ethernet) can fix the MTU size; or systems (such as point-to-point serial links) can determine the MTU at connection time.

[0057] Taking Ethernet as an example, for Ethernet frames transmitted at the link layer, the maximum frame size is usually 1518 bytes, of which 18 bytes are the overhead of the Ethernet header and frame check sequence (FCS), and the rest is the IP MTU, which is the maximum IP data packet that an Ethernet frame can carry, usually 1500 bytes.

[0058] See also Figure 1C , Figure 1C This is a schematic diagram of the structure of an Ethernet frame provided in an embodiment of this application, as shown below. Figure 1C As shown, in the MTU, the IP header occupies 20 bytes and the TCP header occupies 20 bytes, so the user data (or payload, or TCP maximum segment size, MSS) can occupy 1460 bytes.

[0059] As described in the background section, Ethernet frames use the Maximum Transmission Unit (MTU) for data transmission, which improves data transmission efficiency. However, the likelihood of each IP packet occupying exactly one full MTU is very low. In most cases, when IP packets are much smaller than the MTU, each IP packet needs an Ethernet header added at the data link layer before being sent out, resulting in wasted Ethernet frame transmission. When IP packets are larger than the MTU, the sending device cannot send all IP packets through a single Ethernet frame and needs to fragment the IP packets. Fragmentation affects the transmission efficiency of IP packets and also increases the overhead of additional Ethernet headers and FCS (Frame-Standard Cross) elements.

[0060] Based on this, please refer to Figure 2A , Figure 2A A flowchart of a data transmission acceleration method based on maximum transmission unit (MTB) provided in this application embodiment is shown below. Figure 2A As shown, the method includes the following steps:

[0061] 201. Obtain the MTU of the next device, and determine the size of the fillable space of the current data packet based on the MTU and the space size of the existing data in the current data packet.

[0062] Before sending data to the next device, the current device can know the MTU of the next device, so that the current device can determine the size of the data packets (IP packets) it sends based on the MTU of the next device, thus avoiding the situation where the next device cannot accept the data packets due to the sending of excessively large data packets.

[0063] In existing technologies, it is usually sufficient to determine that the data packet sent by the current device is smaller than the MTU of the next device, so that the data packet can be sent normally. However, in the embodiments of this application, in order to improve the output transmission efficiency, the size of the data packet in the Ethernet frame can be made close to the MTU (less than or equal to the MTU, and when it is less than the MTU, the difference between it and the MTU is less than a preset difference).

[0064] Therefore, in this embodiment, the current device first obtains the MTU of the next device. As described above, this can be obtained according to standards. Assuming the current device and the next device communicate via the Ethernet standard protocol, the MTU of the next device can be a preset size, i.e., 1500 bytes. Assuming the current device and the next device have a point-to-point communication connection, the MTU of the next device can be obtained by the current device from the next device, or a value mutually agreed upon when they establish a point-to-point communication connection. This embodiment does not limit the specific size of the MTU. The following description uses an MTU of 1500 bytes as an example.

[0065] Optionally, determining the size of the paddable space for the current data packet based on the MTU includes: determining the payload of the current data packet based on the MTU; and determining the size of the paddable space for the current data packet based on the difference between the payload of the current data packet and the space size of existing data.

[0066] After obtaining the MTU of the next device, the current device can obtain the space size of the existing data in the current data packet, and thus determine the size of the padding space for the current data packet. For example, if the current device includes data packet 1, and the space size of the existing data in data packet 1 is 1000 bytes, then the remaining space size of the data packet = MTU - IP header - TCP header - existing data space size = 1500 - 40 - 1000 = 460 bytes. Here, MTU - IP header - TCP header = payload, so it can also be described as the remaining space size of the data packet + the existing data space size = payload. The padding space size ≤ the remaining space size of the data packet. When the padding space size = the remaining space size of the data packet, the padding space size can be determined to be 460 bytes. When the padding space size < the remaining space size of the data packet, the remaining space size of the data packet - the padding space size can be further set to ≤ a preset difference, such as 10 bytes, 50 bytes, etc. Assuming the preset difference is 50 bytes, the padding space size can be, for example, 400 bytes.

[0067] 202. Obtain the target prediction dictionary corresponding to the size of the fillable space, wherein the target prediction dictionary is one of a plurality of prediction dictionaries.

[0068] In this embodiment, the size of the paddable space of the current data packet is used to fill the preceding data of the next data packet. See details in [link to relevant documentation]. Figure 2B , Figure 2B This application provides a schematic diagram of a multi-data packet transmission embodiment, such as... Figure 2B As shown, the device is currently expected to send three data packets: packet 1 is the current packet, packet 2 is the next packet after packet 1, and packet 3 is the next packet after packet 2 (i.e., the current packet and the next packet are relative concepts and do not refer to any specific packet). Therefore, when transmitting packet 1, if the available space is determined to be 400 bytes, the first 400 bytes of packet 2 need to be obtained to fill the available space in the MTU of packet 1. Similarly, when transmitting packet 2, the preceding data of packet 3 needs to be obtained to fill the available space in the MTU of packet 2.

[0069] Considering that the network interface card (NIC) has no access permissions to the entire data packet "file," but can only access the currently imported data packet, and the next data packet contains future data from the current packet, it cannot be directly read while processing the current packet. Therefore, we cannot directly borrow previous data from the next data packet. Thus, the previous data for the next data packet should be generated in some way. Random generation should be ruled out because the generated data cannot be guaranteed to be the same as the previous data in the next data packet. Furthermore, if the padded data in the current data packet differs from the previous data in the next data packet, then the current and next data packets will not be considered as completely lossless data transmission, and the data transmitted in this way will be corrupted at the receiving end.

[0070] Based on this, this application proposes using a prediction dictionary to generate the preceding data of the next data packet, which is then used to fill the paddable space of the current data packet, generating the padded current data packet. In this process, the target prediction dictionary corresponding to the size of the paddable space of the current data packet is first obtained. This application provides multiple prediction dictionaries, each corresponding to a different data size. For example, these include 200-byte dictionaries, 400-byte dictionaries, 600-byte dictionaries, and 800-byte dictionaries. If the paddable space size is 200 bytes, then the corresponding target prediction dictionary is a 200-byte dictionary.

[0071] Before obtaining the target prediction dictionary corresponding to the size of the paddable space of the current data packet, multiple prediction dictionaries can be generated first. Optionally, before obtaining the target prediction dictionary corresponding to the paddable space, the method further includes: generating multiple initial prediction dictionaries, each prediction dictionary corresponding to a possible space size, and each prediction dictionary including multiple initial prediction items; processing each of the multiple initial prediction dictionaries to remove invalid initial prediction items from the multiple initial prediction items in each initial prediction dictionary, thereby obtaining multiple prediction dictionaries.

[0072] The possible space sizes corresponding to the initial prediction dictionary are, for example, 200 bytes, 400 bytes, 600 bytes, etc. Each prediction dictionary includes an initial prediction item, which refers to the data encoding method corresponding to that possible space size. Therefore, the number of initial prediction items in each initial prediction dictionary is 2^(N*8), where N is the possible space size corresponding to that initial prediction dictionary.

[0073] For details, please refer to [link / reference]. Figure 2C , Figure 2C A schematic diagram of a prediction dictionary provided in an embodiment of this application is shown below. Figure 2C As shown, the possible space size for prediction dictionary 1 is 100 bytes, and the initial prediction terms are the possible encodings of 100 bytes of data, totaling 2^(100*8) terms. It can be seen that the initial prediction terms included in the initial prediction dictionary are extremely large. Using such an initial prediction dictionary to predict data would first place a very heavy storage and retrieval burden on the current device, and secondly, the prediction efficiency would be very low. In fact, the initial prediction terms in the initial prediction dictionary can be simplified by removing invalid initial prediction terms to obtain a more concise prediction dictionary, thereby reducing the storage and retrieval burden on the current device and improving prediction efficiency.

[0074] Optionally, an invalid initial prediction term satisfies at least one of the following: a repeated prediction term, a prediction term without semantic meaning, or a prediction term with incorrect semantic meaning.

[0075] Invalid initial predictions in the initial prediction dictionary can include duplicate predictions. For example... Figure 2CAs shown, for a 100-byte initial prediction dictionary, there is an initial prediction entry 0000…0001 representing the value 1. However, for prediction dictionaries smaller than 100 bytes, there are also prediction entries representing the value 1. The difference lies in the fact that the symbol 1 in the 100-byte initial prediction dictionary is preceded by 2^(100*8)-1 zeros, while the symbol 1 in the prediction dictionary smaller than 100 bytes is preceded by 2^(N'*8)-1 zeros, where N' is less than 100. However, the meanings are the same. Therefore, for a 100-byte initial prediction dictionary, the initial prediction entry 0000…0001 is a duplicate initial prediction entry and can be deleted from the 100-byte initial prediction dictionary. Additionally, invalid initial prediction entries in the initial prediction dictionary can include semantically meaningless prediction entries. For example, the encoded representation of the initial prediction entry is the string "ooy" (not shown in the figure), but this string has no corresponding semantics and the probability of it being used in the text is extremely low. Therefore, this initial prediction entry can be deleted from the initial prediction dictionary. Alternatively, invalid initial predictions in the initial prediction dictionary may include erroneous semantic predictions. For example, an initial prediction might have the encoded representation string "intensiwe" (not shown in the figure), but this is a misspelled word, and such an initial prediction could be removed from the initial prediction dictionary. Invalid initial predictions in the initial prediction dictionary may also include other types, and the examples above in this application should not be limiting.

[0076] Removing invalid predictions from the initial prediction dictionary can significantly optimize its space usage. For example, a 6-byte initial prediction dictionary contains 2^(6*8) initial predictions, which, after simplification, includes approximately 2^45 (valid) predictions. An 800-byte initial prediction dictionary contains 2^(800*8) initial predictions, which, after simplification, includes approximately 2^(800*8*0.2) predictions.

[0077] As can be seen, in this embodiment of the application, after generating the corresponding initial prediction dictionary according to the data space size, it is simplified by removing invalid initial prediction items. The resulting prediction dictionary can greatly reduce its space occupation, reduce the power consumption of the device in storing and reading the prediction dictionary, and thus improve the efficiency of using the prediction dictionary for data prediction.

[0078] 203. Based on the existing data of the current data packet and the target prediction dictionary, obtain the data that can fill the space. The similarity between the data that can fill the space and the previous data of the next data packet is greater than a preset threshold.

[0079] After obtaining the target prediction dictionary, the data that can fill the space can be predicted based on the existing data in the current data packet and the target prediction dictionary. Optionally, obtaining the data that can fill the space based on the existing data in the current data packet and the target prediction dictionary includes: matching the existing data in the current data packet with the target prediction dictionary to obtain the prediction items corresponding to the existing data, wherein the target prediction dictionary includes multiple prediction items, and the size of the prediction items corresponds to the size of the fillable space; inputting the prediction items into the data prediction model, and obtaining the data that can fill the space based on the prediction result of the prediction model.

[0080] For details, please refer to Figure 2D , Figure 2D A schematic diagram of a prediction process provided for an embodiment of this application, such as... Figure 2D As shown, the process of predicting and obtaining data to fill the space begins by matching the existing data in the current data packet with predicted items in the target prediction dictionary. For example, if the existing data in the data packet is 200 bytes, it can be matched with a 200-byte target prediction dictionary entry, specifically predicted item 'n'. Since both predicted item 'n' and the existing data in the data packet are 200 bytes, the matching rate may reach 100%. However, it's also possible that no predicted item in the target prediction dictionary has a 100% matching rate with the existing data in the data packet. Therefore, a preset matching rate can be set for the existing data in the data packet to match the predicted item in the target prediction dictionary. The preset matching rate can be 90%, 95%, etc.

[0081] In some cases, the size of the existing data in a data packet may not exactly correspond to the size of a prediction dictionary. For example, the existing data might be 110 bytes, but a prediction dictionary of size 110 doesn't exist. In such cases, the existing data can be split into two sizes, and then the target prediction dictionaries corresponding to the two sizes can be obtained separately to match the data of those two sizes. For example, 110 bytes can be split into 100 bytes and 10 bytes, and then used to match the prediction items in the 100-byte and 10-byte target dictionaries, respectively. Alternatively, the prediction dictionary closest in size to the existing data can be used as the target prediction dictionary. For example, if the existing data is 110 bytes, a 100-byte prediction dictionary can be obtained as the target prediction dictionary, and its prediction items can be matched.

[0082] After obtaining the prediction term n that matches the existing data in the data packet, the prediction term n can be input into the data prediction model. Based on the prediction result, the data to fill the space is obtained. Specifically, the data prediction model is trained on a large amount of historical data. The historical data includes training data and validation data. The training data includes preceding and following data; the preceding data is the input data, and the following data is the output data. When the probability that preceding data corresponds to the same output data is greater than a preset probability, the output data can be considered the prediction result of the preceding data. For example, if the training data is 0001111111…010101110, and the preceding data, i.e., the first 100 bytes, is 0001111111…, this preceding data is input into the initial data prediction model. The parameters in the initial data prediction model are adjusted so that the output result of the obtained data prediction model corresponds to the prediction result …010101110, thus completing the training of the data prediction model. Therefore, when a prediction item from the target prediction dictionary is used as output data input into the data prediction model, the obtained prediction result is the possible subsequent data corresponding to that prediction item when it is the preceding data. Correspondingly, the data that the existing data in the data packet may subsequently correspond to can be determined, which is the subsequent data predicted by using the prediction item as the preceding data. This can be used as data to fill the space.

[0083] In this embodiment, the existing data of the data packet is matched with the prediction items in the target prediction dictionary, and then the prediction items are input into the data prediction model to obtain the prediction result. This is because the content of the existing data of the data packet is not fixed, and directly using the existing data of the data packet to input into the data prediction model for prediction results makes the accuracy of the prediction result uncontrollable. On the other hand, the prediction items are relatively fixed data, and inputting them into the data prediction model for prediction results ensures a more accurate prediction result, and further ensures the accuracy of the data obtained for filling the space. At the same time, since the length of the prediction items is relatively fixed, the data prediction model only needs to predict for input data of a specific length, improving the efficiency of data prediction.

[0084] 204. Generate a padded current data packet based on the existing data and the available space in the current data packet, and send the padded current data packet.

[0085] After obtaining the available space data for the current data packet, a padded current data packet can be generated based on the existing data and the available space data. This generation involves appending the available space data to the existing data of the current data packet. In other words, the two data segments are concatenated to generate the padded current data packet. The current device then sends the padded current data packet to the next device, thus completing the data packet transmission.

[0086] As can be seen, in this embodiment, the size of the fillable space in the MTU is determined by obtaining the MTU used for data transmission. Then, a target prediction dictionary is obtained, and the data for the fillable space is predicted based on the target prediction dictionary. Finally, the current data packet is generated based on the existing data and the data for the fillable space, and then sent. Since the current data packet is generated based on the MTU size, this ultimately improves data transmission efficiency and speeds up data transmission.

[0087] Optionally, when the sum of the existing data space size and the available padding space size is less than the MTU, the MTU also includes middleware adding data and sending the padded current data packet, including: sending the padded current data packet and sending indication information, the indication information being used to indicate the existing data space size in the padded current data packet, or also being used to indicate the available padding space size in the padded current data packet; the method further includes: receiving a response message, the response message being used to indicate the existing data space size in the padded current data packet received by the receiving end, or also being used to indicate the available padding space size in the padded current data packet.

[0088] See the above. Figure 1A The network architecture scenario diagram illustrates that when a current device sends a padded data packet to a next device, it may pass through one or more middleware, which may add extra data to the padded data packet. This addition of data may cause the padded data packet to exceed the MTU, resulting in fragmentation and weakening the technical effect of the MTU-based data packet transmission method used in this application embodiment.

[0089] Based on this, the current device can send an indication message to the next device simultaneously or after sending the padded current data packet, indicating the size of the existing data space in the padded current data packet. After receiving the padded current data packet, the next device determines the actual size of the existing data space (excluding the data available for padding), compares the two, and determines whether data loss occurred during the reception process. Then, the next device sends a response message to the current device, indicating the size of the existing data space in the padded current data packet received by the next device. This allows the current device to compare the size of the existing data space in its own padded current data packet with the size of the existing data space in the padded current data packet received by the next device. If they are different, it is possible that existing data was lost during the transmission of the padded current data packet, and the current device can resend the padded current data packet to the next device. If they are the same, no existing data was lost during the transmission of the padded current data packet, and the current device does not need to resend the padded current data packet to the next device. As for the data available for padding, and the data added by the middleware in the padded current data packet, regardless of whether loss occurred, it is not necessary to send it to the next device by resending the padded current data packet. This process ensures that truly important data, i.e., the existing data in the current data packet, can be completely sent to the next device, while other non-critical data, including data that can fill space and data added by middleware, skips the confirmation process, thus improving data transmission efficiency.

[0090] Where possible, the instruction information sent by the current device to the next device is also used to indicate the size of the padding space in the current data packet after padding. That is to say, the current device also needs to confirm whether the data in the padding space has been completely sent to the next device, because the data in the padding space can ensure that the transmission of the current data packet after padding is lossless, and avoid the data packet being corrupted in the next device.

[0091] Where possible, the response message sent by the next device to the current device can also be an acknowledgment message. This acknowledgment message confirms whether the size of the existing data in the padded current data packet sent by the current device is the same as the size of the existing data in the padded current data packet received by the next device (or it can also confirm whether the size of the padded space in the padded current data packet sent by the current device is the same as the size of the padded space in the padded current data packet received by the next device). The current device can then directly decide whether to resend the padded current data packet based on the acknowledgment message without needing to perform a further comparison. This improves the communication efficiency between the current device and the next device.

[0092] With the above Figures 2A to 2D For those with the same corresponding implementation, please refer to Figure 3 , Figure 3This is a schematic diagram of the structure of a data transmission device 300 provided in an embodiment of this application, as shown below. Figure 3 As shown, it includes:

[0093] The acquisition unit 301 is used to acquire the MTU of the next device and determine the size of the fillable space of the current data packet based on the MTU and the space size of the existing data in the current data packet.

[0094] The acquisition unit 301 is also used to acquire a target prediction dictionary corresponding to the size of the fillable space, wherein the target prediction dictionary is one of a plurality of prediction dictionaries;

[0095] The prediction unit 302 is used to obtain data that can fill the space based on the existing data of the current data packet and the target prediction dictionary, wherein the similarity between the data that can fill the space and the previous data of the next data packet is greater than a preset threshold.

[0096] The transceiver unit 303 is used to generate a padded current data packet based on the existing data and the data of the padded space in the current data packet, and to send the padded current data packet.

[0097] In one possible example, the acquisition unit 301 is specifically used to: determine the payload of the current data packet based on the MTU; and determine the size of the fillable space of the current data packet based on the difference between the payload of the current data packet and the space size of the existing data.

[0098] In one possible example, the prediction unit 302 is specifically used to: match the existing data of the current data packet with the target prediction dictionary to obtain the prediction item corresponding to the existing data, wherein the target prediction dictionary includes multiple prediction items and the size of the prediction item corresponds to the size of the fillable space; input the prediction item into the data prediction model, and obtain the data of the fillable space based on the prediction result of the prediction model.

[0099] In one possible example, before obtaining the target prediction dictionary corresponding to the fillable space, the obtaining unit 301 is further configured to: generate multiple initial prediction dictionaries, each prediction dictionary corresponding to a possible space size, and each prediction dictionary including multiple initial prediction items; process each of the multiple initial prediction dictionaries to remove invalid initial prediction items from the multiple initial prediction items in each initial prediction dictionary, thereby obtaining multiple prediction dictionaries.

[0100] In one possible example, an invalid initial prediction satisfies at least one of the following: a duplicate prediction, a prediction without semantic meaning, or a prediction with incorrect semantic meaning.

[0101] In one possible example, transceiver unit 303 is specifically used to: add data with fillable space after the existing data in the current data packet to generate the current data packet with filler.

[0102] In one possible example, the MTU also includes middleware adding data. The transceiver unit 303 is specifically used to: send the padded current data packet and send indication information, which is used to indicate the size of the space of the existing data in the padded current data packet, or to indicate the size of the padded space in the padded current data packet.

[0103] The transceiver unit 303 is also used to: receive a response message, which is used to indicate the size of the existing data in the current data packet after padding received by the next device, or to indicate the size of the padding space in the current data packet after padding.

[0104] Specifically, in this application embodiment, the data acquisition device can be divided into functional units according to the above method example. For example, each function can be divided into its own functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0105] With the above Figures 2A to 2D For those with the same corresponding implementation, please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device 1000 provided in an embodiment of this application, as shown below. Figure 4 As shown: The device includes a processor, a memory, and a communication interface. The processor, memory, and communication interface are interconnected and perform communication between them.

[0106] The memory stores executable program code, and the communication interface is used for wireless communication.

[0107] The processor is used to retrieve executable program code stored in memory and execute some or all of the steps of any of the data transmission acceleration methods based on the maximum transmission unit as described in the above method embodiments, wherein the computer includes an electronic terminal device.

[0108] The memory can be volatile memory such as dynamic random access memory (DRAM) or non-volatile memory such as hard disk drive (HDD). The memory stores a set of executable program code, and the processor calls the executable program code stored in the memory to execute instructions including the following:

[0109] Obtain the MTU of the next device, and determine the size of the padding space for the current data packet based on the MTU and the space size of the existing data in the current data packet; obtain the target prediction dictionary corresponding to the size of the padding space, where the target prediction dictionary is one of multiple prediction dictionaries; obtain the data for the padding space based on the existing data of the current data packet and the target prediction dictionary, where the similarity between the data for the padding space and the previous data of the next data packet is greater than a preset threshold; generate the current data packet based on the existing data of the current data packet and the data for the padding space, and send the current data packet.

[0110] As can be seen, in this embodiment, the size of the fillable space in the MTU is determined by obtaining the MTU used for data transmission. Then, a target prediction dictionary is obtained, and the data for the fillable space is predicted based on the target prediction dictionary. Finally, the current data packet is generated based on the existing data and the data for the fillable space, and then sent. Since the current data packet is generated based on the MTU size, this ultimately improves data transmission efficiency and speeds up data transmission.

[0111] Optionally, the size of the fillable space for the current data packet is determined based on the MTU and the space size of the existing data in the current data packet, including: determining the payload of the current data packet based on the MTU; and determining the size of the fillable space for the current data packet based on the difference between the payload of the current data packet and the space size of the existing data.

[0112] Optionally, the data to fill the space is obtained based on the existing data of the current data packet and the target prediction dictionary, including: matching the existing data of the current data packet with the target prediction dictionary to obtain the prediction items corresponding to the existing data, wherein the target prediction dictionary includes multiple prediction items, and the size of the prediction items corresponds to the size of the fillable space; inputting the prediction items into the data prediction model, and obtaining the data to fill the space based on the prediction results of the prediction model.

[0113] Optionally, before obtaining the target prediction dictionary corresponding to the fillable space, the method further includes: generating multiple initial prediction dictionaries, each prediction dictionary corresponding to a possible space size, and each prediction dictionary including multiple initial prediction items; processing each of the multiple initial prediction dictionaries to remove invalid initial prediction items from the multiple initial prediction items in each initial prediction dictionary, thereby obtaining multiple prediction dictionaries.

[0114] Optionally, an invalid initial prediction term satisfies at least one of the following: a repeated prediction term, a prediction term without semantic meaning, or a prediction term with incorrect semantic meaning.

[0115] Optionally, generating a padded current data packet based on the existing data and the data of the padded space in the current data packet includes: adding the data of the padded space after the existing data in the current data packet to generate the padded current data packet.

[0116] Optionally, the MTU also includes middleware adding data and sending the padded current data packet, including: sending the padded current data packet and sending indication information, the indication information being used to indicate the space size of the existing data in the padded current data packet, or also being used to indicate the size of the padded space in the padded current data packet;

[0117] The method further includes: receiving a response message, which indicates the size of the space of existing data in the current data packet after padding received by the next device, or further indicates the size of the padding space in the current data packet after padding.

[0118] This application provides a computer program product, which includes a computer program operable to enable the computer to perform some or all of the steps of any of the maximum transmission unit-based data transmission acceleration methods described in the above method embodiments. The computer program product may be a software installation package.

[0119] This application provides a computer-readable storage medium storing a computer program for performing some or all of the steps of any of the data transmission acceleration methods based on the maximum transmission unit described in the above embodiments.

[0120] It should be noted that, for the sake of simplicity, each of the aforementioned embodiments of the data transmission acceleration method based on the maximum transmission unit is described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0121] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce a good effect.

[0122] Those skilled in the art will understand that all or part of the steps in the various method embodiments of any of the above-described methods for accelerating data transmission based on the maximum transmission unit can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0123] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principle and implementation of a data transmission acceleration method and apparatus based on the maximum transmission unit (MTB). The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​a data transmission acceleration method and apparatus based on the MTB. Therefore, the content of this specification should not be construed as a limitation of this application.

[0124] This application is described with reference to flowchart illustrations and / or block diagrams of methods, hardware products, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0127] It is understood that any product that is controlled or configured to execute the processing method of the flowchart described in the method embodiment of the data transmission acceleration method based on the maximum transmission unit of this application, such as the terminal and computer program product of the above flowchart, falls within the scope of the related products described in this application.

[0128] Obviously, those skilled in the art can make various modifications and variations to the data transmission acceleration method and apparatus based on the maximum transmission unit provided in this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A data transmission acceleration method based on Maximum Transmission Unit (MTU), characterized in that, The method includes: Obtain the MTU of the next device, and determine the size of the fillable space for the current data packet based on the MTU and the space size of the existing data in the current data packet; Obtain the target prediction dictionary corresponding to the size of the fillable space, wherein the target prediction dictionary is one of a plurality of prediction dictionaries; The data for the fillable space is obtained based on the existing data of the current data packet and the target prediction dictionary, wherein the similarity between the data for the fillable space and the previous data of the next data packet is greater than a preset threshold. Generate a padded current data packet based on the existing data in the current data packet and the data in the padded space, and send the padded current data packet; The step of obtaining the data for the fillable space based on the existing data of the current data packet and the target prediction dictionary includes: matching the existing data of the current data packet with the target prediction dictionary to obtain the prediction item corresponding to the existing data, wherein the target prediction dictionary includes multiple prediction items, and the size of the prediction item corresponds to the size of the fillable space; inputting the prediction item into a data prediction model, and obtaining the data for the fillable space based on the prediction result of the prediction model.

2. The method according to claim 1, characterized in that, The step of determining the size of the fillable space for the current data packet based on the MTU and the space size of the existing data in the current data packet includes: The payload of the current data packet is determined based on the MTU. The size of the fillable space for the current data packet is determined based on the difference between the payload of the current data packet and the space size of the existing data.

3. The method according to claim 1, characterized in that, Before obtaining the target prediction dictionary corresponding to the fillable space, the method further includes: Multiple initial prediction dictionaries are generated, each prediction dictionary corresponding to a possible space size, and each prediction dictionary includes multiple initial prediction items; Each of the plurality of initial prediction dictionaries is processed to remove invalid initial prediction items from the plurality of initial prediction items in each initial prediction dictionary, thereby obtaining a plurality of prediction dictionaries.

4. The method according to claim 3, characterized in that, The invalid initial prediction term satisfies at least one of the following: repeated prediction term, no semantic prediction term, or incorrect semantic prediction term.

5. The method according to any one of claims 1-4, characterized in that, The step of generating the padded current data packet based on the existing data of the current data packet and the data of the padded space includes: The data in the fillable space is added after the existing data in the current data packet to generate the filled current data packet.

6. The method according to claim 1, characterized in that, The MTU also includes middleware-added data, and sending the padded current data packet includes: Send the padded current data packet and send indication information, the indication information being used to indicate the space size of the existing data in the padded current data packet, or also to indicate the size of the padded space in the padded current data packet; The method further includes: A response message is received, which is used to indicate the size of the existing data in the current data packet after padding received by the next device, or to indicate the size of the padding space in the current data packet after padding.

7. A data transmission device, characterized in that, The device includes: The acquisition unit is used to acquire the MTU of the next device and determine the size of the fillable space of the current data packet based on the MTU and the space size of the existing data in the current data packet. The acquisition unit is further configured to acquire a target prediction dictionary corresponding to the size of the fillable space, wherein the target prediction dictionary is one of a plurality of prediction dictionaries; The prediction unit is used to obtain the data of the fillable space based on the existing data of the current data packet and the target prediction dictionary, wherein the similarity between the data of the fillable space and the previous data of the next data packet is greater than a preset threshold. The transceiver unit is used to generate a padded current data packet based on the existing data of the current data packet and the data of the padded space, and to send the padded current data packet. The step of obtaining the data for the fillable space based on the existing data of the current data packet and the target prediction dictionary includes: matching the existing data of the current data packet with the target prediction dictionary to obtain the prediction item corresponding to the existing data, wherein the target prediction dictionary includes multiple prediction items, and the size of the prediction item corresponds to the size of the fillable space; inputting the prediction item into a data prediction model, and obtaining the data for the fillable space based on the prediction result of the prediction model.

8. A computer-readable storage medium storing a computer program for electronic data interchange, wherein, The computer program causes the computer to execute the instructions of the steps described in any one of claims 1-6.

9. An electronic device, characterized in that, The device includes: The processor, the memory, and the communication interface are interconnected and perform communication between them. The memory stores executable program code, and the communication interface is used for wireless communication. The processor is configured to retrieve the executable program code stored in the memory and execute the method as described in any one of claims 1-6.

Images