Data transmission method and device, electronic equipment, readable storage medium and program product

Abstract

The application relates to a data transmission method and device, electronic equipment, readable storage medium and program product. A TCP transmission layer receives data from a network through a receiving buffer, identifies the data size of the data received by the receiving buffer, and performs compression processing on the data received by the receiving buffer to obtain compressed data when it is determined that the data size is greater than a set threshold, and then sends the compressed data to a next node of a transmission path. Since the embodiment dynamically reads the data size of the data received by the receiving buffer at this time, and compresses and transmits the data received by the receiving buffer at this time as a complete content block, the content data size does not need to be obtained by analyzing a private protocol, the dependence on the overall content size is reduced, the data demarcation and block problem of the compression algorithm is solved, the compression transmission of the TCP transmission layer data is realized, whether to compress is controlled through the set threshold, and the flexibility and efficiency of the compression transmission of the TCP transmission layer are improved.

Description

Technical Field

[0001] This application relates to the field of data transmission technology, and in particular to a data transmission method, apparatus, electronic device, readable storage medium, and program product. Background Technology

[0002] With the development of data transmission technology, compressing data before transmission can not only significantly improve transmission efficiency, but also greatly save valuable bandwidth resources of CDN (Content Delivery Network).

[0003] In traditional technologies, CDNs, in their Layer 4 (Transport Layer, located at Layer 4 of the OSI (Open System Interconnect) model, primarily responsible for end-to-end data transmission and communication connection management) acceleration under dynamic transmission, do not yet support compressed transmission of uplink and downlink data. This is because Layer 4 acceleration accelerates the content of each customer's custom protocols, and with thousands of customers, it's impossible to parse each one individually. Parsing custom private protocols just to determine content size would significantly slow down the overall response time. Therefore, Layer 4 acceleration cannot determine the boundaries and size of the data, thus making it impossible to compress and transmit the data content.

[0004] However, when faced with large file transfer scenarios, especially during peak periods, uncompressed data can consume a lot of valuable bandwidth resources and increase response time, thereby reducing overall transmission efficiency. Summary of the Invention

[0005] Therefore, it is necessary to provide a data transmission method, apparatus, electronic device, computer-readable storage medium, and computer program product that can improve transmission efficiency in response to the above-mentioned technical problems.

[0006] In a first aspect, this application provides a data transmission method applied to the TCP transport layer, the method comprising:

[0007] By receiving data from the network through the receive buffer, the size of the data received in the receive buffer is identified;

[0008] If the data size is determined to be greater than a set threshold, the data received in the receiving buffer is compressed to obtain compressed data.

[0009] The compressed data is sent to the next node in the transmission path.

[0010] In one embodiment, the method further includes: upon receiving data from the network, storing the received data in a first address space of the receiving buffer; storing the compressed data in a second address space of the receiving buffer; upon determining that a memory mapping list exists, writing the mapping relationship between the first address space and the second address space into the memory mapping list; and upon determining that the compressed data was successfully sent, updating the cache offset of the receiving buffer and updating the memory mapping list according to the first address space and the second address space.

[0011] In one embodiment, the method further includes: creating the memory mapping list if it is determined that no memory mapping list exists; and writing the mapping relationship between the first address space and the second address space into the memory mapping list.

[0012] In one embodiment, the method further includes: updating the cache offset of the receive buffer based on the successfully sent compressed data; if it is determined that the cache space of the receive buffer has reached a receive space threshold, performing the step of receiving data from the network through the receive buffer; if it is determined that the cache space of the receive buffer has not reached the receive space threshold, returning to the step of updating the cache offset of the receive buffer based on the successfully sent compressed data.

[0013] In one embodiment, the method further includes: if it is determined that the data size is less than or equal to a set threshold, sending the data received in the receive buffer to the next node in the transmission path.

[0014] In one embodiment, the step of compressing the data received in the receiving buffer to obtain compressed data includes: compressing the data received in the receiving buffer, determining a compression header and a compression trailer, generating a protocol header including a compression identifier, and obtaining compressed data.

[0015] Secondly, this application provides a data transmission apparatus applied to the TCP transport layer, the apparatus comprising:

[0016] The data receiving module is used to receive data from the network through a receiving buffer and to identify the data size of the data received in the receiving buffer;

[0017] The data compression module is used to compress the data received in the receiving buffer when the data size is determined to be greater than a set threshold, so as to obtain compressed data;

[0018] The data transmission module is used to send the compressed data to the next node in the transmission path.

[0019] Thirdly, this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0020] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0021] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0022] The aforementioned data transmission methods, apparatus, electronic devices, computer-readable storage media, and computer program products all involve the TCP transport layer receiving data from the network through a receive buffer and identifying the size of the data received in the receive buffer. If the data size exceeds a set threshold, the data received in the receive buffer is compressed to obtain compressed data, which is then sent to the next node in the transmission path. Because this embodiment dynamically reads the size of the data received in the receive buffer at that time and compresses and transmits the data as a complete content block, it avoids parsing private protocols to obtain the content data size, reducing dependence on the overall content size. This solves the data delimitation and block segmentation problem of compression algorithms, achieving compressed transmission of TCP transport layer data. By setting a threshold to control whether compression is performed, the flexibility and efficiency of TCP transport layer compressed transmission are improved. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a data transmission method in one embodiment;

[0025] Figure 2 This is a schematic diagram of a data transmission architecture in one embodiment;

[0026] Figure 3 This is a flowchart illustrating the data transmission method in another embodiment;

[0027] Figure 4 This is a structural block diagram of a data transmission device in one embodiment;

[0028] Figure 5 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] By compressing data, data transmission speed and user experience can be improved without increasing bandwidth costs, thus gaining an advantage in a highly competitive market. However, in traditional technologies, CDN's Layer 4 under dynamic transmission cannot compress data content during transmission. Layer 7 (application layer) acceleration under CDN dynamic acceleration, on the other hand, utilizes the compression / decompression modules of NGINX software (a high-performance HTTP (Hypertext Transfer Protocol) server and reverse proxy server used to improve website performance, optimize resource usage, and enhance website security). With simple configuration, compressed transmission can be achieved, thereby improving transmission efficiency. However, the compression transmission function of Layer 7 acceleration cannot be directly applied to Layer 4 acceleration for the following reasons:

[0031] First, in practical applications, Layer 7 acceleration only applies to HTTP / HTTPS protocols. Its acceleration principle involves obtaining the data size and structure from the HTTP header, accurately locating the insertion points for the compression header and trailer, performing streaming compression or decompression on the intermediate data, and finally transmitting the compressed data to the TCP transport layer. Layer 4 acceleration, on the other hand, processes continuous byte streams. Since Layer 4 acceleration involves various custom protocols, parsing these custom protocols to obtain the content size would significantly slow down the overall response time. Conversely, not parsing the protocols makes it difficult to obtain the content size, thus the compression algorithm cannot accurately determine the data boundaries and size, making direct compression impossible. Furthermore, Layer 7 acceleration either compresses all content or leaves all content uncompressed, and only supports traditional downlink compression (i.e., content download scenarios), not uplink compression (i.e., content upload scenarios). This cannot meet the needs of Layer 4 clients in certain scenarios.

[0032] Secondly, in the four-layer acceleration, the receive buffer and the send buffer operate independently and concurrently, lacking a coordination mechanism and making it difficult to achieve balance between inflow and outflow. If the send buffer is slow while the receive buffer is fast, the receive buffer will continuously receive data, eventually filling up. The triggering process will then fail to meet the conditions and cannot enter the send module to send data, leading to a failure to update the receive buffer offset in a timely manner. This creates a mutually dependent locking problem, ultimately causing blocking. This issue limits further improvements in data transmission efficiency, especially in scenarios requiring efficient processing of large-scale data transmissions and real-time data.

[0033] Based on this, this application provides a data transmission method applied to the TCP (Transmission Control Protocol) transport layer, i.e., layer four in the OSI model. This method modifies and optimizes layer seven accelerated compression transmission to meet the compression transmission requirements of layer four acceleration, shortening data transmission time within CDN nodes to achieve efficient data transmission and improved acceleration, thus enhancing customer online experience satisfaction. Through continuous optimization and improvement, CDN can better address data transmission challenges, improve user experience, reduce costs, and achieve more efficient network acceleration. This not only helps enterprises gain an advantage in a highly competitive market but also provides users with faster and more stable network services. Furthermore, with continuous technological advancements, layer four acceleration under dynamic CDN acceleration is expected to achieve even greater breakthroughs in data compression transmission, further improving the overall acceleration effect.

[0034] In one embodiment, such as Figure 1 As shown, a data transmission method is provided, applied to the TCP transport layer, which is the fourth layer in the OSI model. The OSI model defines a seven-layer framework for network interconnection, including, from bottom to top, the physical layer, data link layer, network layer, transport layer, session layer, presentation layer, and application layer. In this embodiment, the method may include the following steps:

[0035] Step 102: Receive data from the network through the receive buffer and identify the data size received in the receive buffer.

[0036] The receive buffer is a memory area in the TCP transport layer used for temporary data storage. In this embodiment, the TCP transport layer receives data from the network through the receive buffer, identifies the size of the data received each time, and then compresses and transmits the data based on subsequent steps.

[0037] Step 104: If the data size is determined to be greater than the set threshold, the data received in the receiving buffer is compressed to obtain compressed data.

[0038] The threshold can be a pre-defined minimum size of data that needs to be compressed. In this embodiment, the TCP transport layer compresses and transmits data based on the relationship between the data size received in the receive buffer and the set threshold identified in the above steps. Specifically, if the TCP transport layer determines that the data size is greater than the set threshold, it compresses the data received in the receive buffer to obtain compressed data.

[0039] Step 106: Send compressed data to the next node in the transmission path.

[0040] Specifically, after obtaining the compressed data based on the above steps, the TCP transport layer sends the compressed data to the next node in the transmission path, thereby achieving compressed data transmission.

[0041] In the aforementioned data transmission method, the TCP transport layer receives data from the network through a receive buffer and identifies the size of the data received in the receive buffer. If the data size exceeds a set threshold, the data received in the receive buffer is compressed to obtain compressed data, which is then sent to the next node in the transmission path. Because this embodiment dynamically reads the size of the data received in the receive buffer at that time and compresses and transmits the data as a complete content block, it avoids parsing private protocols to obtain the content data size, reducing dependence on the overall content size. This solves the data delimitation and block segmentation problem of compression algorithms, achieving compressed transmission of TCP transport layer data. By setting a threshold to control whether compression is performed, the flexibility and efficiency of TCP transport layer compressed transmission are improved.

[0042] In an exemplary embodiment, after identifying the data size of the data received in the receiving buffer in step 102, the method may further include: if the data size is determined to be less than or equal to a set threshold, sending the data received in the receiving buffer to the next node of the transmission path.

[0043] In this embodiment, the TCP transport layer compresses and transmits data based on the relationship between the size of the received data in the receive buffer and a set threshold. Specifically, since the TCP transport layer receives data over a network, and network fluctuations are inevitable, when network fluctuations cause the received data to be too small, compression can be omitted to balance compression resources and transmission efficiency. Therefore, if the TCP transport layer determines that the size of the received data is less than or equal to the set threshold, it will not compress it and will directly send the data received in the receive buffer to the next node in the transmission path. This allows for better adaptation to network fluctuations and changes in the size of the received data. By transmitting the data directly when it is small, compression time can be saved, and transmission efficiency can be improved.

[0044] In an exemplary embodiment, in step 104, the data received in the receiving buffer is compressed to obtain compressed data. Specifically, this may include: compressing the data received in the receiving buffer, determining the compression header and compression tail, generating a protocol header including a compression identifier, and obtaining compressed data.

[0045] The compression header and footer refer to the beginning and end of a compressed file, respectively. They contain essential information to aid in decompression, crucial for proper file extraction. The protocol header is a specific section of data in the compressed file that describes the compressed data format and attributes. It contains necessary metadata that allows decompression software to correctly identify and process the compressed data. The compression identifier indicates whether the current data block has been compressed.

[0046] Since the TCP transport layer in this embodiment transmits both compressed and uncompressed data, a compression flag indicating whether the content is compressed can be added to the internal custom protocol header before data transmission. If compression is performed, the compression header and compression trailer can be determined during the compression process, allowing corresponding nodes to distinguish data boundaries and transmit data transparently. The origin node can determine whether the content is compressed based on the compression flag in the protocol header; if compressed, it decompresses the data; otherwise, it transmits it directly back to the origin. Furthermore, the above processing also applies to downlink compressed transmission, and the data processing logic is consistent with the uplink, making the Layer 4 acceleration function more complete and competitive.

[0047] In one exemplary embodiment, the compressed transport architecture of the TCP transport layer can be as follows: Figure 2 As shown, the edge node, first parent node, and second parent node are all CDN nodes in the TCP transport layer, and the origin station is the target node for TCP transmission. Taking the second parent node as an example of a node returning to the origin. Specifically, during uplink transmission, the edge node first performs compression judgment and corresponding processing on the user's uplink data and transmits it to the first parent node. After receiving the data, the first parent node can pass the data through to the second parent node. Upon receiving the data, the second parent node can determine whether it is compressed data based on the compression flag in the data protocol header. If compressed, it decompresses the data and transmits it to the origin station; otherwise, it directly passes it through to the origin station. The processing procedure for downlink transmission is the same as for uplink, and will not be described further in this embodiment.

[0048] The above embodiments compress data in blocks based on the size of the content data read from the receiving buffer each time, making compressed transmission more flexible and adaptable to different content sizes. Compared to fixed-size compressed blocks, this method can dynamically adjust the compression strategy, improving transmission performance. Furthermore, by adding a compression flag indicating whether the content is compressed to the internal custom protocol header, it enables boundary transmission and reception between different blocks. This helps the parent node and the origin node accurately determine the compression status of the data content, allowing the origin node to completely decompress the compressed data blocks and restore the original data before sending it to the client's origin server. This improves the accuracy and efficiency of data transmission and, compared to traditional unmarked transmission methods, better handles the mixed transmission of compressed and uncompressed data.

[0049] In the Layer 4 acceleration process for raw data reception and compressed data transmission, a problem arises where the receive buffer continuously receives data, but network jitter slows down the transmission speed, easily leading to a full receive buffer and data transmission blockage. Specifically, when the receive buffer is full, the offset update needs to be performed in the sending module. However, the sending module only updates the transmission offset corresponding to the compressed data block, not the buffer offset corresponding to the raw data reception. A full receive buffer triggers a retry waiting operation, preventing further updates to the offset in the sending module, resulting in delayed updates. The underlying reason is that after compressed block transmission, the next-hop CDN node needs to distinguish between data boundaries corresponding to different compressed blocks for reception; otherwise, it will receive incomplete compressed headers and tails, leading to decompression failure. When it is determined that the current receive buffer does not have enough space to receive the next compressed block, the receive buffer chooses not to receive and continues to wait. Once the other compressed content has been sent, it does not have the opportunity to acquire CPU (Central Processing Unit) processing time to update the receive buffer offset, resulting in a problem similar to mutual dependency locks. This causes the entire data transmission to be blocked and wait for a timeout before disconnecting, which in turn leads to the abnormal interruption of the request.

[0050] Based on this, this embodiment of the application solves the above problems by introducing a receive buffer and establishing a mapping relationship management mechanism between the received raw data and compressed data. Specifically, the fundamental reason why the offset of the receive buffer cannot be updated in a timely manner is that the compressed data corresponding to some of the raw data in the buffer has been sent, while the sending module only updates the offset of the compressed data block, but cannot find the corresponding raw data, i.e., the receive buffer data block, to update the offset. Therefore, this embodiment releases the raw data in this part of the receive buffer to ensure that the receive buffer has enough space to receive the next data block, thereby avoiding the blocking problem of requests. To solve this problem, this embodiment introduces a first-in-first-out memory-mapped linked list to store the mapping relationship between the raw data to be compressed and the compressed data blocks corresponding to the receive buffer. During each operation, this linked list is checked to see if any of the compressed data blocks have been sent. Once a compressed block that has been sent is found, the corresponding receive buffer data block is found and its space is released. By resetting the original data blocks in the corresponding receive buffer based on the completion status of the compressed data blocks, the relationship between the original and compressed data in the receive buffer can be accurately balanced and coordinated, avoiding lock problems such as mutual dependence, thereby greatly improving the request response time.

[0051] In one exemplary embodiment, such as Figure 3 As shown, the above data transmission method may further include:

[0052] Step 302: If data is received from the network, the received data is stored in the first address space of the receive buffer.

[0053] The first address space is used to characterize the buffer offset corresponding to the received raw data stored in the receive buffer. In this embodiment, when the TCP transport layer receives data from the network, it stores the received raw data (i.e., uncompressed data) in the first address space of the receive buffer.

[0054] Step 304: Determine whether the data in the first address space is greater than a set threshold.

[0055] The threshold is a pre-defined minimum size of data that needs to be compressed. Specifically, the TCP transport layer determines whether the size of the data received and stored in the first address space is greater than the threshold. If the size of the data in the first address space is greater than the threshold, step 306 is executed; if the size of the data in the first address space is less than or equal to the threshold, step 316 is executed.

[0056] Step 306: Compress the data in the first address space to obtain compressed data.

[0057] Specifically, if the TCP transport layer determines that the size of the data in the first address space exceeds a set threshold, it compresses the data in the first address space to obtain compressed data. The specific compression process can be described in the above embodiment, and will not be repeated here.

[0058] Step 308: Store the compressed data in the second address space of the receive buffer.

[0059] The second address space is used to characterize the cache offset corresponding to the compressed data (i.e., the data obtained after compressing the received original data) stored in the receive buffer. In this embodiment, the TCP transport layer compresses the received original data and stores the compressed data in the second address space of the receive buffer.

[0060] Step 310: Determine if a memory-mapped linked list exists.

[0061] The memory-mapped linked list is used to store the mapping relationship between the raw data to be compressed (i.e., raw data) received in the receive buffer and the compressed data blocks (i.e., compressed data). Specifically, after the TCP transport layer compresses the received raw data and stores it in the second address space of the receive buffer, it further determines whether a memory-mapped linked list has been created. Specifically, if it is determined that a memory-mapped linked list does not exist (i.e., has not been created), step 312 is executed. If it is determined that a memory-mapped linked list exists (i.e., has been created), step 314 is executed.

[0062] Step 312: Create a memory-mapped linked list.

[0063] Specifically, if the TCP transport layer determines that a memory-mapped linked list does not exist (i.e. has not been created), it creates the memory-mapped linked list and initializes it.

[0064] Step 314: Write the mapping relationship between the first address space and the second address space into the memory mapping linked list.

[0065] Since the raw data received by the aforementioned receive buffer and the corresponding compressed data essentially represent the same data, there is a certain mapping relationship between the first address space of the raw data received by the receive buffer and the second address space of the corresponding compressed data. Based on this, the TCP transport layer writes the mapping relationship between the first address space of the raw data received by the receive buffer and the second address space of the corresponding compressed data into a memory-mapped linked list, thereby enabling better management of the receive buffer.

[0066] Step 316: Send data.

[0067] Specifically, the TCP transport layer can send the compressed data to the next node in the transmission path. Alternatively, if the TCP transport layer determines that the size of the data in the first address space is less than or equal to a set threshold, it can directly send the received raw data to the next node in the transmission path. This allows for dynamic adjustment of the transmission strategy and improves transmission performance.

[0068] Step 318: Check if the compressed data was sent successfully.

[0069] Specifically, after the TCP transport layer performs the above steps to send data, it can further detect whether the compressed data has been sent successfully. If it is determined that the compressed data has been sent successfully, it executes step 320. If it is determined that the compressed data has failed to be sent, it returns to execute step 302.

[0070] Step 320: Update the cache offset of the receive buffer according to the first address space and the second address space, and update the memory mapping linked list.

[0071] Specifically, upon confirming successful transmission of compressed data, the TCP transport layer can first update the receive buffer offset based on the second address space, and then update the receive buffer offset based on the first address space. This releases the receive buffer, ensuring it has sufficient space to receive the next data block and preventing request blocking. Furthermore, the TCP transport layer can update the memory mapping list; for example, it can remove the mapping relationship between the first address space of the received raw data and the corresponding second address space of the compressed data from the memory mapping list.

[0072] In the above embodiments, by introducing a receive buffer, since the memory address where the original data is stored and the memory address where the compressed data is stored are in different memory spaces, a memory mapping linked list is established to manage the mapping relationship between the received original data and the compressed data. After the compressed data has been sent, the offset of the corresponding original data buffer can be updated quickly, efficiently and accurately to release its memory space. This avoids the lock problem caused by the receive buffer not having space to receive the next data block, and greatly improves the request response time and transmission efficiency.

[0073] In an exemplary embodiment, the method further includes: updating the cache offset of the receive buffer based on the successfully sent compressed data, and, if it is determined that the cache space of the receive buffer has reached the receive space threshold, performing the step of receiving data from the network through the receive buffer; and, if it is determined that the cache space of the receive buffer has not reached the receive space threshold, returning to the step of updating the cache offset of the receive buffer based on the successfully sent compressed data.

[0074] The receive space threshold can be the minimum space required for the receive buffer to receive data once. In this embodiment, before each data reception, the TCP transport layer updates the receive buffer offset, i.e., releases more memory space, to ensure sufficient space for data reception. Data will only be received normally if the receive buffer space reaches the receive space threshold (i.e., it is determined that there is enough space to receive data). If the receive buffer space does not reach the receive space threshold (i.e., it is determined that there is insufficient space to receive data), the process returns to updating the receive buffer offset. Specifically, the process of updating the receive buffer offset can be referred to the above embodiment, and will not be repeated here.

[0075] In this embodiment, by using a last-in-first-out memory-mapped linked list to store the mapping relationship between the original uncompressed data and the compressed data blocks corresponding to the receive buffer, and by resetting the original data blocks of the corresponding receive buffer according to the completion status of the compressed data blocks, the retry waiting operation and offset update problem caused by the full receive buffer are avoided, thereby ensuring the continuity and stability of data transmission and better meeting the requirements of Layer 4 acceleration.

[0076] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0077] Based on the same inventive concept, this application also provides a data transmission apparatus for implementing the data transmission method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, specific limitations in one or more data transmission apparatus embodiments provided below can be found in the limitations of the data transmission method described above, and will not be repeated here.

[0078] In one exemplary embodiment, such as Figure 4 As shown, a data transmission device is provided, which is applied to the TCP transport layer. The device includes: a data receiving module 402, a data compression module 404, and a data transmission module 406, wherein:

[0079] The data receiving module 402 is used to receive data from the network through a receiving buffer and to identify the data size of the data received in the receiving buffer;

[0080] The data compression module 404 is used to compress the data received in the receiving buffer when it is determined that the data size is greater than a set threshold, so as to obtain compressed data;

[0081] The data transmission module 406 is used to send the compressed data to the next node in the transmission path.

[0082] In an exemplary embodiment, the apparatus further includes a memory mapping module, configured to: upon receiving data from the network, store the received data in a first address space of the receive buffer; store the compressed data in a second address space of the receive buffer; if a memory mapping list is determined to exist, write the mapping relationship between the first address space and the second address space into the memory mapping list; and if the compressed data is determined to have been successfully sent, update the cache offset of the receive buffer according to the first address space and the second address space, and update the memory mapping list.

[0083] In an exemplary embodiment, the memory mapping module is further configured to: create the memory mapping linked list if it is determined that no memory mapping linked list exists; and write the mapping relationship between the first address space and the second address space into the memory mapping linked list.

[0084] In an exemplary embodiment, the memory mapping module is further configured to: update the cache offset of the receive buffer based on the successfully sent compressed data; if it is determined that the cache space of the receive buffer has reached the receive space threshold, execute the receiving of data from the network through the receive buffer; if it is determined that the cache space of the receive buffer has not reached the receive space threshold, return to the execution of updating the cache offset of the receive buffer based on the successfully sent compressed data.

[0085] In an exemplary embodiment, the data transmission module is further configured to: send the data received in the receiving buffer to the next node of the transmission path if it is determined that the data size is less than or equal to a set threshold.

[0086] In an exemplary embodiment, the data compression module is further configured to: compress the data received in the receiving buffer, determine the compression header and compression tail, generate a protocol header including a compression identifier, and obtain compressed data.

[0087] Each module in the aforementioned data transmission device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0088] In one exemplary embodiment, a computer device is provided, which may be a server applied to the TCP transport layer, and its internal structure diagram may be as follows. Figure 5 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data for transmission. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a data transmission method.

[0089] Those skilled in the art will understand that Figure 5 The structure shown is only a block diagram of a part of the structure related to the present application and does not constitute a limitation on the computer device on which the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0090] In one exemplary embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0091] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0092] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0093] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0094] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0095] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0096] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data transmission method, characterized in that, The method is applied to the TCP transport layer, and the method includes: By receiving data from the network through the receive buffer, the size of the data received in the receive buffer is identified; If the data size is determined to be greater than a set threshold, the data received in the receiving buffer is compressed to obtain compressed data. The compressed data is sent to the next node in the transmission path; When data is received from the network, the received data is stored in the first address space of the receive buffer; The compressed data is stored in the second address space of the receiving buffer; If it is determined that a memory mapping list exists, the mapping relationship between the first address space and the second address space is written into the memory mapping list; If the compressed data is successfully sent, the cache offset of the receiving buffer is updated according to the first address space and the second address space, and the memory mapping list is also updated. Update the buffer offset of the receive buffer based on the successfully sent compressed data; If it is determined that the buffer space of the receiving buffer has reached the receiving space threshold, the step of receiving data from the network through the receiving buffer is executed. If it is determined that the buffer space of the receive buffer has not reached the receive space threshold, the process returns to the step of updating the buffer offset of the receive buffer based on the successfully sent compressed data.

2. The method according to claim 1, characterized in that, The method further includes: If it is determined that the memory-mapped linked list does not exist, the memory-mapped linked list shall be created. The mapping relationship between the first address space and the second address space is written into the memory mapping linked list.

3. The method according to any one of claims 1 to 2, characterized in that, The method further includes: If the data size is determined to be less than or equal to a set threshold, the data received in the receive buffer is sent to the next node in the transmission path.

4. The method according to any one of claims 1 to 2, characterized in that, The step of compressing the data received in the receiving buffer to obtain compressed data includes: The data received in the receive buffer is compressed, a compression header and a compression tail are determined, a protocol header including a compression identifier is generated, and compressed data is obtained.

5. A data transmission device, characterized in that, The apparatus is applied to the TCP transport layer, and the apparatus includes: The data receiving module is used to receive data from the network through a receiving buffer and to identify the data size of the data received in the receiving buffer; The data compression module is used to compress the data received in the receiving buffer when the data size is determined to be greater than a set threshold, so as to obtain compressed data; A data transmission module is used to send the compressed data to the next node in the transmission path; When data is received from the network, the received data is stored in the first address space of the receive buffer; The compressed data is stored in the second address space of the receiving buffer; If it is determined that a memory mapping list exists, the mapping relationship between the first address space and the second address space is written into the memory mapping list; If the compressed data is successfully sent, the cache offset of the receiving buffer is updated according to the first address space and the second address space, and the memory mapping list is also updated. Update the buffer offset of the receive buffer based on the successfully sent compressed data; If it is determined that the buffer space of the receiving buffer has reached the receiving space threshold, the step of receiving data from the network through the receiving buffer is executed. If it is determined that the buffer space of the receive buffer has not reached the receive space threshold, the process returns to the step of updating the buffer offset of the receive buffer based on the successfully sent compressed data.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images