Data transmission method and device, electronic equipment, readable storage medium and program product

Abstract

The invention relates to a data transmission method and device, electronic equipment, a readable storage medium and a program product. The TCP transmission layer receives data from a network through the receiving cache region, identifies the data size of the data received by the receiving cache region, compresses the data received by the receiving cache region to obtain compressed data under the condition that the data size is determined to be greater than a set threshold value, and further sends the compressed data to a next node of a transmission path. According to the embodiment of the invention, the size of the currently received data of the receiving cache region is dynamically read, and the currently received data of the receiving cache region is compressed and transmitted as a complete content block, so that the size of the content data does not need to be acquired by analyzing a private protocol, the dependence on the overall content size is reduced, and the data transmission efficiency is improved. The problem of data demarcation partitioning of a compression algorithm is solved, compression transmission of TCP transmission layer data is achieved, whether compression is conducted or not is controlled by setting a threshold value, and the flexibility and efficiency of TCP transmission layer compression transmission are improved.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to a data transmission method, device, electronic device, readable storage medium and program product. Background Art

[0002] With the development of data transmission technology, compressing data before transmission can not only significantly improve transmission efficiency, but also greatly save the precious bandwidth resources of CDN (Content Delivrey Network).

[0003] In traditional technology, CDN does not support compressed transmission of upstream and downstream data in terms of Layer 4 acceleration (the transport layer is located at the fourth layer in the OSI (Open System Interconnect) model and is mainly responsible for end-to-end data transmission and communication connection management) under dynamic transmission. This is because Layer 4 acceleration accelerates the content of customized protocols of each customer, and there are thousands of customers who cannot customize the parsing one by one. If the customized private protocol is parsed to obtain the content size, it will greatly drag down the overall response time. Therefore, Layer 4 acceleration cannot determine the boundaries and size of the data, and thus cannot compress and transmit the data content.

[0004] However, when facing some large file transfer business scenarios, especially during business peak hours, uncompressed data will occupy a lot of valuable bandwidth resources and also increase response time, thereby reducing overall transmission efficiency. Summary of the invention

[0005] Based on this, it is necessary to provide a data transmission method, device, electronic device, computer-readable storage medium and computer program product that can improve transmission efficiency in response to the above technical problems.

[0006] In a first aspect, the present application provides a data transmission method, which is applied to a TCP transport layer, and the method includes:

[0007] Receiving data from the network through a receiving buffer, and identifying the data size of the data received by the receiving buffer;

[0008] When it is determined that the data size is greater than a set threshold, compressing the data received by the receiving buffer to obtain compressed data;

[0009] The compressed data is sent to a next node in the transmission path.

[0010] In one of the embodiments, the method further includes: when data is received from a network, storing the received data in a first address space of the receiving buffer; storing the compressed data in a second address space of the receiving buffer; when it is determined that a memory mapping linked list exists, writing a mapping relationship between the first address space and the second address space into the memory mapping linked list; when it is determined that the compressed data is sent successfully, updating a cache offset of the receiving buffer according to the first address space and the second address space, and updating the memory mapping linked list.

[0011] In one of the embodiments, the method further includes: when it is determined that the memory mapping linked list does not exist, creating the memory mapping linked list; and writing a mapping relationship between the first address space and the second address space into the memory mapping linked list.

[0012] In one of the embodiments, the method further includes: updating the cache offset of the receiving buffer area based on the compressed data that has been successfully sent; when it is determined that the cache space of the receiving buffer area reaches the receiving space threshold, executing the step of receiving data from the network through the receiving buffer area; when it is determined that the cache space of the receiving buffer area does not reach the receiving space threshold, returning to executing the step of updating the cache offset of the receiving buffer area based on the compressed data that has been successfully sent.

[0013] In one of the embodiments, the method further includes: when it is determined that the data size is less than or equal to a set threshold, sending the data received by the receiving buffer to the next node of the transmission path.

[0014] In one of the embodiments, compressing the data received by the receiving buffer to obtain compressed data includes: compressing the data received by the receiving buffer, determining a compression header and a compression footer, generating a protocol header including a compression identifier, and obtaining compressed data.

[0015] In a second aspect, the present application provides a data transmission device, which is applied to a TCP transport layer, and includes:

[0016] A data receiving module, used for receiving data from a network through a receiving buffer area, and identifying a data size of the data received by the receiving buffer area;

[0017] A data compression module, configured to compress the data received by the receiving buffer area to obtain compressed data when it is determined that the data size is greater than a set threshold;

[0018] The data transmission module is used to send the compressed data to the next node of the transmission path.

[0019] In a third aspect, the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0021] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0022] In the above-mentioned data transmission method, device, electronic device, computer-readable storage medium and computer program product, the TCP transport layer receives data from the network through the receiving buffer area, and identifies the data size of the data received in the receiving buffer area. When it is determined that the data size is greater than the set threshold, the data received in the receiving buffer area is compressed to obtain compressed data, and then the compressed data is sent to the next node of the transmission path. Since this embodiment dynamically reads the data size of the data received in the receiving buffer area at that time, and compresses and transmits the data received in the receiving buffer area at that time as a complete content block, it is not necessary to parse the private protocol to obtain the content data size, which reduces the dependence on the overall content size, solves the data boundary and block problem of the compression algorithm, realizes the compressed transmission of TCP transport layer data, and controls whether to compress by setting a threshold, thereby improving the flexibility and efficiency of the compressed transmission of the TCP transport layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of a flow chart of a data transmission method in an embodiment;

[0025] Figure 2 A schematic diagram of a data transmission architecture in one embodiment;

[0026] Figure 3 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0027] Figure 4 is a structural block diagram of a data transmission device in one embodiment;

[0028] Figure 5 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] By compressing data, data transmission speed and user experience can be improved without increasing bandwidth costs, thus gaining an advantage in a highly competitive market. However, in traditional technologies, CDN cannot compress data content for transmission at the fourth layer under dynamic transmission. With the seventh layer (application layer) acceleration under CDN dynamic acceleration, compression transmission can be achieved based on simple configuration by applying the compression / decompression function module of NGINX software (a high-performance HTTP (Hypertext Transfer Protocol) server and reverse proxy server used to improve website performance, optimize resource usage, and enhance website security), thereby improving transmission efficiency. However, the compression transmission function of the seventh layer acceleration cannot be directly applied to the fourth layer acceleration for the following reasons:

[0031] First, in actual applications, the seven-layer acceleration is only applicable to the HTTP / HTTPS protocol. Its acceleration principle is to obtain the data size and structure from the HTTP protocol header, and then accurately locate the insertion position of the compression header and compression tail, perform streaming compression or decompression operations on the intermediate data, and finally pass the compressed data to the TCP transport layer for transmission. The four-layer acceleration processes a continuous byte stream. Since the four-layer acceleration involves various custom protocols, if the custom private protocol is parsed to obtain the content size, it will greatly drag down the overall response time, and it will be difficult to obtain the content size without parsing the protocol. Therefore, the compression algorithm cannot accurately determine the boundaries and size of the data, and thus cannot directly perform compression processing. In addition, the seven-layer acceleration either compresses all content or does not compress all content, and only supports traditional downlink compression (i.e., content download scenario), but not uplink compression (i.e., content upload scenario), which cannot meet the needs of four-layer customers in some scenarios.

[0032] Secondly, in the four-layer acceleration, the receiving buffer and the sending buffer perform receiving and sending operations independently and concurrently, lacking a coordination mechanism, making it difficult to achieve inbound and outbound balance. If the sending buffer is slow and the receiving buffer is fast, this will cause the receiving buffer to continuously receive data, eventually causing the buffer to be full. If the triggering process does not meet the conditions, it will not be able to enter the sending module to send data, causing the offset of the receiving buffer to fail to be updated in time, forming a mutually dependent lock problem, and ultimately leading to blocking. This problem limits the further improvement of data transmission efficiency, especially in scenarios that require efficient processing of large-scale data transmission and real-time data.

[0033] Based on this, an embodiment of the present application provides a data transmission method, which is applied to the TCP (Transmission Control Protocol) transport layer, i.e., the fourth layer in the OSI model. It meets the compression transmission requirements of the fourth layer acceleration by performing transformation and optimization on the basis of the seven-layer accelerated compression transmission, shortening the data transmission time within the CDN node, so as to achieve efficient data transmission and improve the acceleration effect, and better improve customer Internet experience satisfaction. Through continuous optimization and improvement, CDN can better cope with the challenges of data transmission, improve user experience, reduce costs, and achieve more efficient network acceleration. This not only helps enterprises gain an advantage in the fiercely competitive market, but also provides users with faster and more stable network services. And with the continuous advancement of technology, the four-layer acceleration under CDN dynamic acceleration is expected to achieve greater breakthroughs in data compression transmission, thereby further improving the overall acceleration effect.

[0034] In one embodiment, Figure 1 As shown, a data transmission method is provided, which is applied to the TCP transport layer, i.e., the fourth layer in the OSI model, wherein the OSI model defines a seven-layer framework of network interconnection, including from bottom to top the physical layer, data link layer, network layer, transport layer, session layer, presentation layer, and application layer. In this embodiment, the method may include the following steps:

[0035] Step 102: Receive data from the network through the receiving buffer, and identify the data size of the data received by the receiving buffer.

[0036] The receive buffer is a memory area in the TCP transport layer for temporarily storing data. In this embodiment, the TCP transport layer receives data from the network through the receive buffer, identifies the size of each received data in the receive buffer, and then compresses and transmits the data based on subsequent steps.

[0037] Step 104 , when it is determined that the data size is greater than a set threshold, compress the data received in the receiving buffer to obtain compressed data.

[0038] The set threshold value may be a preset minimum value of the data size that needs to be compressed. In this embodiment, the TCP transport layer compresses and transmits the data based on the relationship between the data size of the receiving buffer area identified in the above steps and the set threshold value. Specifically, when the TCP transport layer determines that the data size is greater than the set threshold value, it compresses the data received in the receiving buffer area to obtain compressed data.

[0039] Step 106: Send the compressed data to the next node of the transmission path.

[0040] Specifically, after obtaining the compressed data based on the above steps, the TCP transport layer sends the compressed data to the next node of the transmission path, thereby realizing compressed transmission of the data.

[0041] In the above data transmission method, the TCP transport layer receives data from the network through the receiving buffer area, and identifies the data size of the data received in the receiving buffer area. When it is determined that the data size is greater than the set threshold, the data received in the receiving buffer area is compressed to obtain compressed data, and then the compressed data is sent to the next node of the transmission path. Since the present embodiment dynamically reads the data size of the data received in the receiving buffer area at that time, and compresses and transmits the data received in the receiving buffer area at that time as a complete content block, it is not necessary to parse the private protocol to obtain the content data size, and reduces the dependence on the overall content size, so as to solve the data boundary and block problem of the compression algorithm, realize the compressed transmission of the TCP transport layer data, and control whether to compress by setting a threshold, thereby improving the flexibility and efficiency of the compressed transmission of the TCP transport layer.

[0042] In an exemplary embodiment, in step 102, after identifying the data size of the data received by the receiving buffer, the method may further include: when it is determined that the data size is less than or equal to a set threshold, sending the data received by the receiving buffer to the next node of the transmission path.

[0043] In this embodiment, the TCP transport layer compresses and transmits data based on the relationship between the data size of the data received in the identified receiving buffer area and the set threshold. Specifically, since the TCP transport layer receives data in the network, and network fluctuations are inevitable, when the network fluctuations cause the data received at that time to be too small, in order to balance the compression resources and transmission efficiency, it may not be compressed. Based on this, when the TCP transport layer determines that the data size of the received data at that time is less than or equal to the set threshold, it does not compress and directly sends the data received in the receiving buffer area to the next node of the transmission path. This can better adapt to changes in network fluctuations and the size of received data. By directly transmitting when the received data is small, it can not only save compression time, but also improve transmission efficiency.

[0044] In an exemplary embodiment, in step 104, the data received by the receiving buffer is compressed to obtain compressed data, which may specifically include: compressing the data received by the receiving buffer, determining a compression header and a compression footer, generating a protocol header including a compression identifier, and obtaining compressed data.

[0045] The compression header and compression footer refer to the beginning and end of the compressed file. They contain the necessary information to assist in decompression, which is crucial for correctly decompressing the file. The protocol header is a specific piece of data used to describe the format and properties of compressed data in the compressed file. The protocol header contains the necessary meta information so that the decompression software can correctly identify and process the compressed data. The compression flag is used to indicate whether the current data block has been compressed.

[0046] Since the content transmitted by the TCP transport layer in this embodiment includes compressed data and uncompressed data, a compression mark indicating whether the content is compressed can be added to the internal custom protocol header, and then data transmission can be performed. If compression is performed, the compression header and compression tail can be determined during the compression process so that the corresponding node can distinguish the data boundary and perform transparent transmission. The return source node can determine whether the content is compressed based on the compression mark of the protocol header. If it is compressed, it will be decompressed. If it is not compressed, it will be directly transparently transmitted back to the source. The above processing process is also applicable to downlink compressed transmission, and the data processing logic is consistent with the uplink, making the four-layer acceleration function more complete and competitive.

[0047] In an exemplary embodiment, the compression transport architecture of the TCP transport layer can be as follows: Figure 2 As shown, the edge node, the first parent node and the second parent node are all CDN nodes of the TCP transport layer, and the source station is the target node of the TCP transmission, wherein the second parent node is taken as the source node for example. Specifically, during uplink transmission, the user's uplink data is first compressed and judged and processed accordingly by the edge node, and transmitted to the first parent node. After receiving the data, the first parent node can transparently transmit the data to the second parent node. After receiving the data, the second parent node can determine whether it is compressed data based on the compression identifier in the data protocol header. If it is compressed, it is decompressed and transmitted to the source station. If it is not compressed, it is directly transparently transmitted to the source station. The processing process during downlink transmission is the same as that of the uplink, and this embodiment will not be repeated.

[0048] The above embodiment compresses by reading the content data size in the receiving buffer area each time, making the compressed transmission more flexible and adaptable to different content sizes. Compared with the fixed-size compression block method, this method can dynamically adjust the compression strategy and improve the transmission performance. And by adding a compression mark to indicate whether the content is compressed in the internal custom protocol header to achieve the boundary transmission and reception between different blocks, it helps the parent node and the source node to accurately judge the compression status of the data content, so that when returning to the source node, the compressed data block can be completely decompressed to restore the original data and send it to the customer source station, which improves the accuracy and efficiency of data transmission. Compared with the traditional unidentified transmission method, it can better handle the mixed transmission of compressed and uncompressed data.

[0049] Because the receiving buffer area continuously receives data content in the four-layer accelerated original data reception and compressed data transmission link, and the transmission speed slows down due to network jitter, which easily causes the receiving buffer area to be full, resulting in data transmission and transmission blocking. Specifically, when the receiving buffer area is full, since the offset update of the receiving buffer area needs to be performed in the sending module, the sending module actually only updates the sending offset corresponding to the compressed data block, and does not update the buffer area offset corresponding to the original data reception. The full receiving buffer area will trigger a retry waiting operation, and it is impossible to enter the sending module again to update the offset, resulting in the receiving buffer area offset not being updated in time. The specific reason is that after the compressed block transmission, the next-hop CDN node needs to distinguish the data boundaries corresponding to different compressed blocks for reception, otherwise it will receive incomplete compressed headers and compression tails, resulting in decompression failure. When it is determined that the current receiving buffer area does not have enough space to receive the next compressed block, the receiving buffer area chooses not to receive and continues to wait. After the other compressed content has been sent, there is no opportunity to obtain CPU (Central Processing Unit) processing time to update the receive buffer offset, resulting in a similar interdependent lock problem, which blocks the entire data transmission and waits for timeout, resulting in abnormal request interruption.

[0050] Based on this, the embodiment of the present application solves the above problem by introducing a receiving buffer and establishing a mapping relationship management mechanism between the received original data and the compressed data. Specifically, the fundamental reason why the offset of the receiving buffer cannot be updated in time is that the compressed data corresponding to some of the original data in the buffer has been sent, and the sending module only updates the offset of the compressed data block, but cannot find its corresponding original data, that is, the receiving buffer data block to update the offset. Therefore, the present embodiment releases the original data of this part of the receiving buffer to ensure that the receiving buffer has enough space to receive the next data block, thereby avoiding the blocking problem of the request. In order to solve this problem, the present embodiment introduces a first-in-last-out memory mapping linked list to store the mapping relationship between the original data to be compressed and the compressed data block corresponding to the receiving buffer. Each time the operation is performed, this linked list will be checked to see whether all the compressed data blocks have been sent. Once it is found that there is a compressed block that has been sent, the corresponding receiving buffer data block will be found and its space will be released. By resetting the original data blocks in the corresponding receiving buffer according to the completion of the sending of the compressed data blocks, the relationship between the original data and the compressed data in the receiving buffer can be accurately balanced and coordinated, avoiding similar interdependent lock problems, thereby greatly improving the request response time.

[0051] In an exemplary embodiment, Figure 3 As shown, the above data transmission method may further include:

[0052] Step 302: When data is received from the network, the received data is stored in the first address space of the receiving buffer.

[0053] The first address space is used to represent the cache offset corresponding to the original data received by the receiving buffer. In this embodiment, when the TCP transport layer receives data from the network, it stores the received original data (ie, uncompressed data) in the first address space of the receiving buffer.

[0054] Step 304, determining whether the data in the first address space is greater than a set threshold.

[0055] The set threshold is a preset minimum value of the data size that needs to be compressed. Specifically, the TCP transport layer determines whether the size of the data received and stored in the first address space is greater than the set threshold, and if it is determined that the size of the data in the first address space is greater than the set threshold, step 306 is executed, and if it is determined that the size of the data in the first address space is less than or equal to the set threshold, step 316 is executed.

[0056] Step 306: compress the data in the first address space to obtain compressed data.

[0057] Specifically, when the TCP transport layer determines that the size of the data in the first address space is greater than a set threshold, the data in the first address space is compressed to obtain compressed data. The specific compression process can be the same as that in the above embodiment, which will not be described in detail in this embodiment.

[0058] Step 308: Store the compressed data in the second address space of the receiving buffer.

[0059] The second address space is used to represent the cache offset corresponding to the compressed data stored in the receiving buffer (i.e., the data obtained by compressing the original data received above). In this embodiment, the TCP transport layer compresses the original data received above and stores the compressed data in the second address space of the receiving buffer.

[0060] Step 310, determine whether there is a memory mapping linked list.

[0061] The memory mapping linked list is used to store the mapping relationship between the original data to be compressed (i.e., original data) received by the receiving buffer and the compressed data block (i.e., compressed data). Specifically, after the TCP transport layer compresses the received original data and stores it in the second address space of the receiving buffer, it further determines whether the memory mapping linked list has been created. Specifically, if it is determined that the memory mapping linked list does not exist (i.e., has not been created), step 312 is executed. If it is determined that the memory mapping linked list exists (i.e., has been created), step 314 is executed.

[0062] Step 312, create a memory mapping linked list.

[0063] Specifically, when the TCP transport layer determines that the memory mapping linked list does not exist (ie, has not been created), it creates a memory mapping linked list and initializes the table.

[0064] Step 314: write the mapping relationship between the first address space and the second address space into a memory mapping linked list.

[0065] Since the original data received by the receiving buffer and the corresponding compressed data substantially represent the same data, there is a certain mapping relationship between the first address space of the original data received by the receiving buffer and the second address space of the corresponding compressed data. Based on this, the TCP transport layer writes the mapping relationship between the first address space of the original data received by the receiving buffer and the second address space of the corresponding compressed data into the memory mapping linked list, so as to better manage the receiving buffer.

[0066] Step 316, sending data.

[0067] Specifically, the TCP transport layer can send the compressed data to the next node of the transmission path. Alternatively, when the TCP transport layer determines that the size of the data in the first address space is less than or equal to the set threshold, it can directly send the received original data to the next node of the transmission path. This allows dynamic adjustment of the transmission strategy and improves transmission performance.

[0068] Step 318, checking whether the compressed data is sent successfully.

[0069] Specifically, after executing the above steps to send data, the TCP transport layer may further detect whether the compressed data is sent successfully, and execute step 320 if it is determined that the compressed data is sent successfully, and return to execute step 302 if it is determined that the compressed data fails to be sent.

[0070] Step 320: Update the cache offset of the receiving buffer area according to the first address space and the second address space, and update the memory mapping linked list.

[0071] Specifically, when the TCP transport layer determines that the compressed data is sent successfully, it can first update the cache offset of the receiving buffer area according to the second address space, and then update the cache offset of the receiving buffer area according to the first address space, thereby releasing the receiving buffer area to ensure that the receiving buffer area has enough space to receive the next data block to avoid the problem of request blocking. In addition, the TCP transport layer can also update the memory mapping linked list, for example, the mapping relationship between the first address space of the received original data and the second address space of the corresponding compressed data can be deleted in the memory mapping linked list.

[0072] In the above embodiment, by introducing a receiving buffer, since the memory address location where the original data is stored and the memory address location where the compressed data is stored are in different location spaces, therefore, by establishing a mapping relationship management mechanism between the received original data and the compressed data, namely a memory mapping linked list, after the compressed data has been sent, the corresponding original data buffer offset can be updated quickly, efficiently and accurately to release its memory space, thereby avoiding the waiting lock problem caused by the lack of space in the receiving buffer to receive the next data block, greatly improving the request response time and transmission efficiency.

[0073] In an exemplary embodiment, the above method also includes: updating the cache offset of the receiving buffer area based on the compressed data that has been successfully sent, and when it is determined that the cache space of the receiving buffer area reaches the receiving space threshold, executing the step of receiving data from the network through the receiving buffer area; when it is determined that the cache space of the receiving buffer area does not reach the receiving space threshold, returning to execute the step of updating the cache offset of the receiving buffer area based on the compressed data that has been successfully sent.

[0074] Among them, the receiving space threshold can be the minimum space required for the receiving buffer to receive data once. In this embodiment, before receiving data each time, the TCP transport layer will update the cache offset of the receiving buffer, that is, release more memory space to ensure that there is enough space to receive data. Only when it is determined that the cache space of the receiving buffer reaches the receiving space threshold (that is, it is determined that there is enough space to receive data), the data will be received normally. If it is determined that the cache space of the receiving buffer does not reach the receiving space threshold (that is, it is determined that there is not enough space to receive data), it returns to continue updating the cache offset of the receiving buffer. Specifically, the specific process of updating the cache offset of the receiving buffer can refer to the above embodiment, and this embodiment will not be repeated.

[0075] In this embodiment, a first-in-last-out memory mapping linked list is used to store the mapping relationship between the original data to be compressed and the compressed data blocks corresponding to the receiving buffer, and the original data blocks corresponding to the receiving buffer are reset according to the completion status of the sending of the compressed data blocks, thereby avoiding the retry waiting operation and offset update problems caused by the full receiving buffer, thereby ensuring the continuity and stability of data transmission and better meeting the four-layer acceleration requirements.

[0076] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0077] Based on the same inventive concept, the embodiment of the present application also provides a data transmission device for implementing the data transmission method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data transmission device embodiments provided below can refer to the limitations on the data transmission method above, and will not be repeated here.

[0078] In an exemplary embodiment, Figure 4 As shown, a data transmission device is provided, which is applied to the TCP transport layer. The device includes: a data receiving module 402, a data compression module 404 and a data transmission module 406, wherein:

[0079] The data receiving module 402 is used to receive data from the network through the receiving buffer area and identify the data size of the data received by the receiving buffer area;

[0080] A data compression module 404 is used to compress the data received by the receiving buffer area to obtain compressed data when it is determined that the data size is greater than a set threshold;

[0081] The data transmission module 406 is configured to send the compressed data to the next node of the transmission path.

[0082] In an exemplary embodiment, the device also includes a memory mapping module, which is used to: when data is received from the network, store the received data in the first address space of the receiving buffer area; store the compressed data in the second address space of the receiving buffer area; when it is determined that there is a memory mapping linked list, write the mapping relationship between the first address space and the second address space into the memory mapping linked list; when it is determined that the compressed data is sent successfully, update the cache offset of the receiving buffer area according to the first address space and the second address space, and update the memory mapping linked list.

[0083] In an exemplary embodiment, the memory mapping module is further used to: create the memory mapping linked list when it is determined that the memory mapping linked list does not exist; and write the mapping relationship between the first address space and the second address space into the memory mapping linked list.

[0084] In an exemplary embodiment, the memory mapping module is also used to: update the cache offset of the receiving buffer area based on the compressed data that has been successfully sent; when it is determined that the cache space of the receiving buffer area reaches the receiving space threshold, execute receiving data from the network through the receiving buffer area; when it is determined that the cache space of the receiving buffer area does not reach the receiving space threshold, return to execute updating the cache offset of the receiving buffer area based on the compressed data that has been successfully sent.

[0085] In an exemplary embodiment, the data transmission module is further configured to: when it is determined that the data size is less than or equal to a set threshold, send the data received by the receiving buffer to the next node of the transmission path.

[0086] In an exemplary embodiment, the data compression module is further used to: compress the data received by the receiving buffer area, determine the compression header and the compression tail, generate a protocol header including a compression identifier, and obtain compressed data.

[0087] Each module in the above data transmission device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0088] In an exemplary embodiment, a computer device is provided. The computer device may be a server and applied to a TCP transport layer. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store and transmit data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data transmission method is implemented.

[0089] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0090] In an exemplary embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0091] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0092] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0093] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0094] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0095] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0096] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data transmission method, characterized in that: The method is applied to the TCP transport layer, and the method comprises: Receiving data from the network through a receiving buffer, and identifying the data size of the data received by the receiving buffer; When it is determined that the data size is greater than a set threshold, compressing the data received by the receiving buffer to obtain compressed data; The compressed data is sent to a next node in the transmission path.

2. The method according to claim 1, characterized in that The method further comprises: In case of receiving data from the network, storing the received data in the first address space of the receiving buffer area; storing the compressed data in a second address space of the receiving buffer; When it is determined that a memory mapping linked list exists, writing a mapping relationship between the first address space and the second address space into the memory mapping linked list; When it is determined that the compressed data is sent successfully, the cache offset of the receiving buffer area is updated according to the first address space and the second address space, and the memory mapping linked list is updated.

3. The method according to claim 2, characterized in that The method further comprises: When it is determined that the memory mapping linked list does not exist, creating the memory mapping linked list; The mapping relationship between the first address space and the second address space is written into the memory mapping linked list.

4. The method according to claim 2, characterized in that: The method further comprises: Update the buffer offset of the receiving buffer area based on the compressed data that has been sent successfully; In the case where it is determined that the buffer space of the receiving buffer area reaches the receiving space threshold, performing a step of receiving data from the network through the receiving buffer area; When it is determined that the buffer space of the receiving buffer area does not reach the receiving space threshold, the step of returning to execute updating the buffer offset of the receiving buffer area based on the compressed data that has been sent successfully is performed.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When it is determined that the data size is less than or equal to the set threshold, the data received by the receiving buffer area is sent to the next node of the transmission path.

6. The method according to any one of claims 1 to 4, characterized in that: The compressing the data received by the receiving buffer area to obtain compressed data includes: The data received by the receiving buffer area is compressed, a compression header and a compression tail are determined, a protocol header including a compression identifier is generated, and compressed data is obtained.

7. A data transmission device, characterized in that: The device is applied to the TCP transport layer, and the device comprises: A data receiving module, used for receiving data from a network through a receiving buffer area, and identifying a data size of the data received by the receiving buffer area; A data compression module, configured to compress the data received by the receiving buffer area to obtain compressed data when it is determined that the data size is greater than a set threshold; The data transmission module is used to send the compressed data to the next node of the transmission path.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images