Online transmission method and system based on block data variable compression

Abstract

The invention discloses an online transmission method and system based on block data variable compression, and the method comprises the steps: a first client obtains user data, compresses the user data, and transmits the compressed user data to a cache block if the online transmission request is data transmission between users after responding to the online transmission request of the data; the cache block can convert the compressed user data into dynamic data, and the data in the cache block is transmitted to the second client after being compressed for the second time; and if the online transmission request is the data transmission between the block users, directly transmitting the compressed user data to the second client. In this way, the storage form of the data can be changed in the transmission process when online transmission of large files is carried out between users, transmission and compression can be carried out at the same time, the transmission space cost is saved, and the data transmission efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of block data transmission, and in particular to an online transmission method and system based on variable compression of block data. Background Art

[0002] With the rapid development of information technology, efficient and secure data transmission has become one of the core challenges of blockchain technology and many network application fields. In the blockchain ecosystem, the transmission of data blocks is not only related to the response speed of the system, but also directly affects the stability and scalability of the entire network. At present, although there are a variety of data transmission technologies, such as online file transmission and face-to-face transmission, these technologies still face a series of significant limitations when applied to block data transmission.

[0003] First, as a widely used data transmission method, online file transfer provides a convenient way to exchange data in the Internet environment. However, when processing large-scale and high-frequency data block transmissions in the blockchain, its efficiency is often restricted by multiple factors such as data size, network bandwidth, and server load. Especially in scenarios with high real-time requirements, the transmission of large amounts of data blocks may increase network latency, affecting user experience and system performance.

[0004] Secondly, although face-to-face transmission (such as through USB, Bluetooth and other short-range wireless technologies) can ensure the security of data transmission to a certain extent, its application scenarios are limited and it is not suitable for extensive, cross-regional block data interaction in blockchain networks. In addition, face-to-face transmission is also difficult to compare with online transmission in high-speed network environments in terms of efficiency, especially when processing large-scale data blocks, its inefficiency and inconvenience are particularly prominent.

[0005] In addition, existing blockchain data transmission involves multiple key links: fast transmission of known block files, efficient data interaction between blocks, and real-time transmission links between blocks and users. These links together constitute the core framework of the blockchain data transmission system. However, when implementing these functions, existing technologies generally face the problem that data transmission efficiency is limited by data status (such as compression) and bandwidth throughput efficiency between users / blocks.

[0006] Specifically, although data compression technology can reduce the amount of transmitted data and improve transmission efficiency to a certain extent, its processing requires additional time and computing resources, and the compression effect is limited by the data type and compression algorithm. At the same time, the bandwidth throughput efficiency between users and between blocks is also a key factor affecting the data transmission speed. In the case of complex and changeable network environment, how to ensure the stability and efficiency of data transmission has become an urgent problem to be solved in the development of blockchain technology. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide an online transmission method and system based on variable compression of block data, which can save transmission space cost and improve data transmission efficiency when large files are transmitted online.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0009] An online transmission method based on variable compression of block data comprises the steps of:

[0010] The first client obtains user data and compresses the user data;

[0011] The first client responds to the online transmission request, and if the online transmission request is data transmission between users, transmits the compressed user data to the cache block, the cache block converts the compressed user data into dynamic data, and performs secondary compression on the data in the cache block before transmitting it to the second client;

[0012] If the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client.

[0013] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0014] An online transmission system based on variable compression of block data includes a first client, a second client and a cache block:

[0015] The first client obtains user data and compresses the user data;

[0016] The first client responds to the online transmission request, and if the online transmission request is data transmission between users, transmits the compressed user data to the cache block, the cache block converts the compressed user data into dynamic data, and performs secondary compression on the data in the cache block before transmitting it to the second client;

[0017] If the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client.

[0018] The beneficial effects of the present invention are as follows: after the first client obtains the user data, it compresses it, and after responding to the online transmission request of the data, if the online transmission request is data transmission between users, the compressed user data is transmitted to the cache block, and the cache block can convert the compressed user data into dynamic data, and perform secondary compression on the data in the cache block before transmitting it to the second client; if the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client. In this way, when large files are transmitted online between users, the storage form of the data can be changed during the transmission process, and the data can be compressed while being transmitted, saving the transmission space cost and improving the data transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of an online transmission method based on variable compression of block data according to an embodiment of the present invention;

[0020] Figure 2 The figure is a schematic diagram of an online transmission system based on variable compression of block data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0022] Please refer to Figure 1 The embodiment of the present invention provides an online transmission method based on variable compression of block data, comprising the steps of:

[0023] The first client obtains user data and compresses the user data;

[0024] The first client responds to the online transmission request, and if the online transmission request is data transmission between users, transmits the compressed user data to the cache block, the cache block converts the compressed user data into dynamic data, and performs secondary compression on the data in the cache block before transmitting it to the second client;

[0025] If the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client.

[0026] From the above description, it can be seen that the beneficial effects of the present invention are: after the first client obtains the user data, it compresses it, and after responding to the online transmission request of the data, if the online transmission request is data transmission between users, the compressed user data is transmitted to the cache block, and the cache block can convert the compressed user data into dynamic data, and perform secondary compression on the data in the cache block before transmitting it to the second client; if the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client. In this way, when large files are transmitted online between users, the storage form of the data can be changed during the transmission process, and it can be compressed while transmitting, saving the transmission space cost and improving the data transmission efficiency.

[0027] Furthermore, the first client obtains user data and compresses the user data, including:

[0028] If the user data obtained by the first client is dynamic data, the user data is segmented, the data of each segment is compressed, and it is determined whether the compression ratio of each segment is greater than a preset ratio. If so, the compressed data is used for transmission, otherwise, the uncompressed data is used for transmission;

[0029] If the user data acquired by the first client is a storage file, the storage file is compressed in the memory of the first client.

[0030] As can be seen from the above description, user data includes dynamic data types and storage file types. Therefore, for dynamic data, since the size of the data is already known in the memory, segmenting the dynamic data can achieve multi-segment filling of data and improve the transmission efficiency of dynamic data; and for storage files, the files can also be compressed for transmission to reduce the amount of data transmission and improve transmission efficiency.

[0031] Furthermore, the cache block converts the compressed user data into dynamic data, including:

[0032] If the compressed user data is compressed dynamic data, the cache block caches the compressed user data into the memory block, and marks the original data state of the compressed user data in the memory block;

[0033] If the compressed user data is a compressed storage file, the cache block caches the compressed user data into the memory block to convert the compressed user data into memory storage data, and marks the original data state of the compressed user data in the memory block.

[0034] From the above description, it can be seen that in the cache block, the data of the stored file type is converted into a dynamic data type for dynamic storage, which can change the data storage format during the transmission process and perform efficient transmission; and the memory block storing the data in the cache block is marked with the original data status corresponding to the compressed user data, so as to facilitate the subsequent second client to decompress and restore the data.

[0035] Furthermore, the data in the cache block is compressed twice, including:

[0036] The cache block compresses the data in the memory block in segments, releases the secondary compressed data into the buffer after generating a buffer, and configures the corresponding original data state for the secondary compressed data.

[0037] From the above description, it can be seen that by using the method of segmented compression of data, the data originally stored in the file can be compressed twice according to the segmented compression method of dynamic data, which can achieve the effect of compressing large files while transmitting. At the same time, the secondary compression method can also further reduce the transmission volume to improve transmission efficiency.

[0038] Furthermore, it also includes:

[0039] The second client receives the compressed data and its original data state, and if the original data state of the received compressed data is dynamic data, creates a data space, decompresses the compressed data in segments, and fills the compressed data into the data space;

[0040] If the original data state of the received compressed data is a storage file, the compressed data is decompressed and stored in the disk space.

[0041] From the above description, it can be seen that after the second client receives the compressed data, for the compressed data whose original data state is dynamic data, the data is decompressed in segments and filled into the data space; for the compressed data whose original data state is a storage file, the data is decompressed and stored in the disk space. In this way, the data transmission efficiency can be improved.

[0042] Please refer to Figure 2 Another embodiment of the present invention provides an online transmission system based on variable compression of block data, including a first client, a second client and a cache block:

[0043] The first client obtains user data and compresses the user data;

[0044] The first client responds to the online transmission request, and if the online transmission request is data transmission between users, transmits the compressed user data to the cache block, the cache block converts the compressed user data into dynamic data, and performs secondary compression on the data in the cache block before transmitting it to the second client;

[0045] If the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client.

[0046] From the above description, it can be seen that the beneficial effects of the present invention are: after the first client obtains the user data, it compresses it, and after responding to the online transmission request of the data, if the online transmission request is data transmission between users, the compressed user data is transmitted to the cache block, and the cache block can convert the compressed user data into dynamic data, and perform secondary compression on the data in the cache block before transmitting it to the second client; if the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client. In this way, when large files are transmitted online between users, the storage form of the data can be changed during the transmission process, and it can be compressed while transmitting, saving the transmission space cost and improving the data transmission efficiency.

[0047] Furthermore, the first client obtains user data and compresses the user data, including:

[0048] If the user data obtained by the first client is dynamic data, the user data is segmented, the data of each segment is compressed, and it is determined whether the compression ratio of each segment is greater than a preset ratio. If so, the compressed data is used for transmission, otherwise, the uncompressed data is used for transmission;

[0049] If the user data acquired by the first client is a storage file, the storage file is compressed in the memory of the first client.

[0050] As can be seen from the above description, user data includes dynamic data types and storage file types. Therefore, for dynamic data, since the size of the data is already known in the memory, segmenting the dynamic data can achieve multi-segment filling of data and improve the transmission efficiency of dynamic data; and for storage files, the files can also be compressed for transmission to reduce the amount of data transmission and improve transmission efficiency.

[0051] Furthermore, the cache block converts the compressed user data into dynamic data, including:

[0052] If the compressed user data is compressed dynamic data, the cache block caches the compressed user data into the memory block, and marks the original data state of the compressed user data in the memory block;

[0053] If the compressed user data is a compressed storage file, the cache block caches the compressed user data into the memory block to convert the compressed user data into memory storage data, and marks the original data state of the compressed user data in the memory block.

[0054] From the above description, it can be seen that in the cache block, the data of the stored file type is converted into a dynamic data type for dynamic storage, which can change the data storage format during the transmission process and perform efficient transmission; and the memory block storing the data in the cache block is marked with the original data status corresponding to the compressed user data, so as to facilitate the subsequent second client to decompress and restore the data.

[0055] Furthermore, the data in the cache block is compressed twice, including:

[0056] The cache block compresses the data in the memory block in segments, releases the secondary compressed data into the buffer after generating a buffer, and configures the corresponding original data state for the secondary compressed data.

[0057] From the above description, it can be seen that by using the method of segmented compression of data, the data originally stored in the file can be compressed twice according to the segmented compression method of dynamic data, which can achieve the effect of compressing large files while transmitting. At the same time, the secondary compression method can also further reduce the transmission volume to improve transmission efficiency.

[0058] Furthermore, it also includes:

[0059] The second client receives the compressed data and its original data state, and if the original data state of the received compressed data is dynamic data, creates a data space, decompresses the compressed data in segments, and fills the compressed data into the data space;

[0060] If the original data state of the received compressed data is a storage file, the compressed data is decompressed and stored in the disk space.

[0061] From the above description, it can be seen that after the second client receives the compressed data, for the compressed data whose original data state is dynamic data, the data is decompressed in segments and filled into the data space; for the compressed data whose original data state is a storage file, the data is decompressed and stored in the disk space. In this way, the data transmission efficiency can be improved.

[0062] The above-mentioned online transmission method and system based on variable compression of block data of the present invention are particularly suitable for saving transmission space cost and improving data transmission efficiency when transmitting large files online. The following is an explanation through specific implementation methods:

[0063] Embodiment 1

[0064] Please refer to Figure 1 , an online transmission system based on variable compression of block data, comprising the steps of:

[0065] S1. A first client obtains user data and compresses the user data.

[0066] Among them, if the user data obtained by the first client is dynamic data, the user data is segmented, the data of each segment is compressed, and it is determined whether the compression ratio of each segment is greater than the preset ratio. If so, the compressed data is used for transmission, otherwise, the uncompressed data is used for transmission.

[0067] Specifically, we can first calculate how many segments the dynamic data can be divided into: number of segments = dynamic data size / 1024, then compress the current segment data into RAR, ZIP or RAR5 format, and then determine the size change between the current segment compressed data and the original data, and calculate the compression ratio by (original data size - compressed data size) / original data size. When the compression ratio > 20%, it is considered to be effective compression, and the compressed data size is recorded at the same time. Otherwise, the uncompressed current segment data is directly transmitted in real time.

[0068] If the user data acquired by the first client is a storage file, the storage file is compressed in the memory of the first client.

[0069] Specifically, the storage file is read into the memory, compressed in the memory, and the size of the data after the memory compression is calculated, and the size of the data after the memory compression is recorded.

[0070] Among them, since the size of dynamic data is already known in the memory and the data has been prepared, it is only necessary to calculate the size of the memory data space and process it in segments. The purpose is: segmented transmission is more efficient. Since the position after segmentation is known, multi-segment filling data operations can be performed.

[0071] For storage files, (time / bandwidth from user data to user data transmission target) < (time / bandwidth from user data compressed data to user data transmission target). If the efficiency of direct transmission of storage files is higher than the efficiency of transmission after compression, real-time transmission is used.

[0072] In this embodiment, please refer to Figure 2 For the compressed data, different compressed data of the same user data can also be compared, and the data with smaller data volume after compression is preferentially selected as the new compressed data.

[0073] S2. The first client responds to the online transmission request. If the online transmission request is data transmission between users, the compressed user data is transmitted to the cache block. The cache block converts the compressed user data into dynamic data, and performs secondary compression on the data in the cache block before transmitting it to the second client.

[0074] If the compressed user data is compressed dynamic data, the cache block caches the compressed user data into a memory block and marks the original data state of the compressed user data in the memory block.

[0075] Specifically, when transmitting between users, it is necessary to set up a cache block between users. For the case where the user-to-block transmission efficiency is greater than the user-to-user transmission efficiency, that is, the efficiency of "user 1->cache block->user 2" is greater than the efficiency of "user 1->user 2", the transmission efficiency can be effectively improved.

[0076] The function of the cache block is to perform secondary processing on the data and data mode: for dynamic data, after obtaining the data size, the compressed user data is cached in the cache memory block, and the original data status is marked for the memory block. At the same time, the data in the memory block is compressed and released to the buffer;

[0077] If the compressed user data is a compressed storage file, the cache block caches the compressed user data into the memory block to convert the compressed user data into memory storage data, and marks the original data state of the compressed user data in the memory block.

[0078] Specifically, for storage files, the compressed user data is converted into memory storage data during the transmission process, so that the data storage form is changed to dynamic data and stored in the cache memory block, and the original data status of the memory block is marked. At this time, secondary compression can be performed according to the segmented compression method of dynamic data, which can achieve the effect of compressing large files while transmitting, and at the same time compress the data in the memory block and release it to the buffer.

[0079] S3. If the online transmission request is data transmission between block users, directly transmit the compressed user data to the second client.

[0080] Among them, the second client receives the compressed data and its original data state. If the original data state of the received compressed data is dynamic data, a data space is created, and the compressed data is decompressed in segments and filled into the data space; if the original data state of the received compressed data is a storage file, the compressed data is decompressed and stored in the disk space.

[0081] Specifically, for compressed data whose original data state is dynamic data, obtain the number of segments of the compressed data to calculate the total size of the dynamic data: total size = number of segments * 1024; create a data space, decompress the corresponding compression format (RAR / ZIP / RAR5) and fill the space, and fill the data into the data space in sequence.

[0082] For compressed data whose original data state is a storage file, the compressed data is decompressed to the disk space.

[0083] Embodiment 2

[0084] Please refer to Figure 2 , an online transmission system based on variable compression of block data, comprising a first client, a second client and a cache block:

[0085] The first client obtains user data and compresses the user data: if the user data obtained by the first client is dynamic data, the user data is segmented, the data of each segment is compressed, and it is determined whether the compression ratio of each segment is greater than a preset ratio. If so, the compressed data is used for transmission, otherwise, the uncompressed data is used for transmission; if the user data obtained by the first client is a storage file, the storage file is compressed in the memory of the first client.

[0086] The first client responds to the online transmission request. If the online transmission request is data transmission between users, the first client transmits the compressed user data to the cache block. The cache block converts the compressed user data into dynamic data, and performs secondary compression on the data in the cache block before transmitting it to the second client. If the compressed user data is compressed dynamic data, the cache block caches the compressed user data into the memory block, and marks the original data state of the compressed user data in the memory block. If the compressed user data is a compressed storage file, the cache block caches the compressed user data into the memory block to convert the compressed user data into memory storage data, and marks the original data state of the compressed user data in the memory block. The secondary compression of the data in the cache block includes: the cache block compresses the data in the memory block in segments, releases the secondary compressed data into the buffer after generating the buffer, and configures the corresponding original data state for the secondary compressed data.

[0087] If the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client.

[0088] The second client receives the compressed data and its original data state. If the original data state of the received compressed data is dynamic data, a data space is created, and the compressed data is decompressed in segments and filled into the data space; if the original data state of the received compressed data is a storage file, the compressed data is decompressed and stored in the disk space.

[0089] In summary, the present invention provides an online transmission method and system based on variable compression of block data. After the first client obtains the user data, it compresses it, and after responding to the online transmission request of the data, if the online transmission request is data transmission between users, the compressed user data is transmitted to the cache block. The cache block can convert the compressed user data into dynamic data, and compress the data in the cache block in segments before transmitting it to the second client. Therefore, by changing the data storage format during the transmission process, the transmission can be changed to compression, and the transmission efficiency can be improved by 13 to 31%. If the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client. The above transmission method can adapt to data transmission in two scenarios.

[0090] In this way, the overall efficiency of transmission is improved due to the reduction of data size after data compression. Since the data link is a long connection, the efficiency is improved, and the number of links in the long connection is reduced as a whole, which improves the overall efficiency. However, big data with too many long links will cause the data processing burden of the server and increase the data processing and transmission time. In this solution, dynamic scenarios are used to generate multiple transmission solutions in combination with multiple situations, effectively saving transmission space costs and solving transmission problems from the perspective of data size.

[0091] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An online transmission method based on variable compression of block data, characterized in that: Includes steps: The first client obtains user data and compresses the user data; The first client responds to the online transmission request, and if the online transmission request is data transmission between users, transmits the compressed user data to the cache block, the cache block converts the compressed user data into dynamic data, and performs secondary compression on the data in the cache block before transmitting it to the second client; If the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client.

2. The online transmission method based on variable compression of block data according to claim 1, characterized in that: The first client obtains user data and compresses the user data, including: If the user data obtained by the first client is dynamic data, the user data is segmented, the data of each segment is compressed, and it is determined whether the compression ratio of each segment is greater than a preset ratio. If so, the compressed data is used for transmission, otherwise, the uncompressed data is used for transmission; If the user data acquired by the first client is a storage file, the storage file is compressed in the memory of the first client.

3. The online transmission method based on variable compression of block data according to claim 1, characterized in that: The cache block converts the compressed user data into dynamic data, including: If the compressed user data is compressed dynamic data, the cache block caches the compressed user data into the memory block, and marks the original data state of the compressed user data in the memory block; If the compressed user data is a compressed storage file, the cache block caches the compressed user data into the memory block to convert the compressed user data into memory storage data, and marks the original data state of the compressed user data in the memory block.

4. The online transmission method based on variable compression of block data according to claim 3 is characterized in that: Performing secondary compression on the data in the cache block includes: The cache block compresses the data in the memory block in segments, releases the secondary compressed data into the buffer after generating a buffer, and configures the corresponding original data state for the secondary compressed data.

5. The online transmission method based on variable compression of block data according to claim 4 is characterized in that: Also includes: The second client receives the compressed data and its original data state, and if the original data state of the received compressed data is dynamic data, creates a data space, decompresses the compressed data in segments, and fills the compressed data into the data space; If the original data state of the received compressed data is a storage file, the compressed data is decompressed and stored in the disk space.

6. An online transmission system based on variable compression of block data, characterized in that: Including the first client, the second client and the cache block: The first client obtains user data and compresses the user data; The first client responds to the online transmission request, and if the online transmission request is data transmission between users, transmits the compressed user data to the cache block, the cache block converts the compressed user data into dynamic data, and performs secondary compression on the data in the cache block before transmitting it to the second client; If the online transmission request is data transmission between block users, the compressed user data is directly transmitted to the second client.

7. The online transmission system based on variable compression of block data according to claim 6, characterized in that: The first client obtains user data and compresses the user data, including: If the user data obtained by the first client is dynamic data, the user data is segmented, the data of each segment is compressed, and it is determined whether the compression ratio of each segment is greater than a preset ratio. If so, the compressed data is used for transmission, otherwise, the uncompressed data is used for transmission; If the user data acquired by the first client is a storage file, the storage file is compressed in the memory of the first client.

8. The online transmission system based on variable compression of block data according to claim 6, characterized in that: The cache block converts the compressed user data into dynamic data, including: If the compressed user data is compressed dynamic data, the cache block caches the compressed user data into the memory block, and marks the original data state of the compressed user data in the memory block; If the compressed user data is a compressed storage file, the cache block caches the compressed user data into the memory block to convert the compressed user data into memory storage data, and marks the original data state of the compressed user data in the memory block.

9. The online transmission system based on variable compression of block data according to claim 8, characterized in that: Performing secondary compression on the data in the cache block includes: The cache block compresses the data in the memory block in segments, releases the secondary compressed data into the buffer after generating a buffer, and configures the corresponding original data state for the secondary compressed data.

10. The online transmission system based on variable compression of block data according to claim 9, characterized in that: Also includes: The second client receives the compressed data and its original data state, and if the original data state of the received compressed data is dynamic data, creates a data space, decompresses the compressed data in segments, and fills the compressed data into the data space; If the original data state of the received compressed data is a storage file, the compressed data is decompressed and stored in the disk space.

Images