A method and system for user data transmission based on a distributed network

Abstract

The application discloses a kind of method and system based on distributed network user data transmission, after receiving the data that first client sends for the first time, it is converted into dictionary format data, to store the common structural features of data by dictionary format data and carry out data compression and simplification.When the second client issues download request to data, download dictionary format data from the server, and restore the original data format by formula with dictionary format data, in this way, the second client decompresses specified data by dictionary, and there is no need to repeat download repeated data during data transmission;And the same structure of data has common structure, dictionary-data storage technology is established for it to realize data "slimming" and data dictionary format conversion, and through data dictionary format conversion instead of data encryption, it is also not necessary to change transmission mode, can improve data transmission efficiency while ensuring the security of data transmission.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, and in particular to a method and system for user data transmission based on a distributed network. Background Technology

[0002] In the current information technology field, with the explosive growth of data volume, data storage and access efficiency has become one of the core challenges in the design of distributed systems. Distributed file systems, as a key technology for handling large-scale data storage and access, significantly improve data reliability and access speed by distributing data across multiple nodes. However, in environments with high-concurrency data requests, traditional distributed file systems still face many bottlenecks in response speed and data processing capabilities.

[0003] Data compression technology plays a crucial role in distributed systems. By reducing the storage space occupied by data, compression technology can not only lower storage costs but also improve the efficiency of data transmission and access to a certain extent. Data compression technology is widely used in mainstream operating systems such as Windows, but most of these applications focus on compressing single files or data blocks, lacking a holistic solution optimized for distributed environments.

[0004] While existing technologies such as Windows data compression solutions perform well in single-node environments, they still have significant shortcomings in distributed file systems:

[0005] 1. Current technologies primarily focus on data compression itself, that is, reducing the physical size of data through algorithms, but often neglect targeted optimization for common file types or data characteristics. This "one-size-fits-all" compression method cannot fully utilize the characteristics of different data types, thus limiting compression efficiency and effectiveness.

[0006] 2. While existing distributed file systems support data compression, they typically only cover the storage of compressed data, lacking effective management and organization of compressed files. This means that the system needs additional time to decompress data during access, thus impacting overall access speed.

[0007] 3. As data security becomes increasingly important, data encryption has become an indispensable function. However, existing technologies often struggle to achieve effective data encryption while ensuring fast data access. This lack of security poses a potential risk to distributed file systems in the storage of sensitive data.

[0008] 4. In high-concurrency data request modes, distributed file systems need to respond quickly to a large number of concurrent requests and provide stable data access services. However, existing data compression and access mechanisms often struggle to meet these requirements simultaneously, leading to a decline in system performance.

[0009] Given the aforementioned background and technical challenges, it is particularly important to develop a data compression and management solution that can be optimized for the characteristics of distributed file systems. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide a method and system for user data transmission based on a distributed network, which can improve data transmission efficiency and ensure data transmission security.

[0011] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0012] A method for user data transmission based on a distributed network includes the following steps:

[0013] The server converts the data received from the first client into dictionary-formatted data, which includes the structural attribute features of the data and the data content features compressed by a formula, and stores the dictionary-formatted data.

[0014] In response to the download request for the data, the second client downloads the dictionary-formatted data from the server and restores the dictionary-formatted data to the original data format using a formula.

[0015] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows:

[0016] A system for user data transmission based on a distributed network includes a first client, a server, and a second client.

[0017] The server converts the data received from the first client into dictionary-formatted data, which includes the structural attribute features of the data and the data content features compressed by a formula, and stores the dictionary-formatted data.

[0018] In response to the download request for the data, the second client downloads the dictionary-formatted data from the server and restores the dictionary-formatted data to the original data format using a formula.

[0019] The beneficial effects of this invention are as follows: After receiving the data initially sent by the first client, the server converts it into dictionary-formatted data to compress and simplify the data by leveraging the shared structural features of the dictionary-formatted data storage. When the second client sends a download request for data, it downloads the dictionary-formatted data from the server and restores it to the original data format using a formula. In this way, the second client decompresses the specified data using a dictionary, eliminating the need to repeatedly download duplicate data during data transmission. Furthermore, data with the same structure share a common structure, establishing a dictionary-data storage technology to achieve data "slimming" and data dictionary format conversion. Since data dictionary format conversion, rather than data encryption, does not require changing the transmission mode, it can improve data transmission efficiency while ensuring data transmission security. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a method for transmitting user data based on a distributed network, according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of a user data transmission system based on a distributed network according to an embodiment of the present invention;

[0022] Figure 3 This is a flowchart illustrating the specific steps of a method for transmitting user data based on a distributed network, according to an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram illustrating dictionary format conversion in a method for transmitting user data based on a distributed network, according to an embodiment of the present invention. Detailed Implementation

[0024] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0025] Please refer to Figure 1 This invention provides a method for user data transmission based on a distributed network, comprising the following steps:

[0026] The server converts the data received from the first client into dictionary-formatted data, which includes the structural attribute features of the data and the data content features compressed by a formula, and stores the dictionary-formatted data.

[0027] In response to the download request for the data, the second client downloads the dictionary-formatted data from the server and restores the dictionary-formatted data to the original data format using a formula.

[0028] As described above, the beneficial effects of this invention are as follows: After receiving the data initially sent by the first client, the server converts it into dictionary-format data to compress and simplify the data by utilizing the shared structural features of the dictionary-format data storage. When the second client sends a download request for data, it downloads the dictionary-format data from the server and restores it to the original data format using a formula. In this way, the second client decompresses the specified data using a dictionary, eliminating the need to repeatedly download duplicate data during data transmission. Furthermore, data with the same structure share a common structure, establishing a dictionary-data storage technology to achieve data "slimming" and data dictionary format conversion. Simultaneously, by using data dictionary format conversion rather than data encryption, and without changing the transmission mode, data transmission efficiency is improved while ensuring data transmission security.

[0029] Furthermore, the server converts the received data from the first client into dictionary-formatted data. This dictionary-formatted data includes structural attribute features and data content features compressed using a formula, including:

[0030] The server obtains data from the first client, uses the data format of the data as the structural attribute feature, and uses the data other than the structural attribute feature as the data content feature;

[0031] If the data content features can be compressed using a formula, then formula-compressed data is obtained; otherwise, a data storage segment is obtained, and the formula-compressed data, the data storage segment, and the structural attribute features are used as dictionary-formatted data.

[0032] As described above, for data content features that cannot be processed by the dictionary, they are directly generated as data storage segments. For data content features that can be processed by the dictionary, corresponding formula compressed data is generated. Therefore, dictionary-formatted data includes not only structural attribute features but also data storage segments and formula compressed data in the data content features, which can handle multiple data types and further improve data transmission efficiency.

[0033] Furthermore, taking the data format of the data as a structural attribute feature, and the data other than the structural attribute feature as a data content feature, the method further includes:

[0034] The structural attribute features and data content features of the data are stored in the corresponding nodes in the form of a binary tree.

[0035] As described above, structural attribute features and data content features are stored in a binary tree to facilitate the subsequent restoration of dictionary-formatted data.

[0036] Furthermore, storing the dictionary-formatted data includes:

[0037] The compressed formula data is segmented and the segmented compressed formula data is stored in a binary tree node.

[0038] As described above, by segmenting data, a dictionary-data-segmentation storage technology can be established to further reduce data size.

[0039] Furthermore, the dictionary-formatted data is restored to its original format using a formula, including:

[0040] Based on the structural attribute features in the dictionary-formatted data, generate the structural information of the corresponding data;

[0041] The compressed formula data in the dictionary format data is restored, and the restored compressed formula data and the data storage segment are filled into the structure information of the data.

[0042] As described above, generating structural information by using structural attribute features from dictionary-formatted data and then filling it with data restored from the dictionary-formatted data can effectively improve data transmission efficiency.

[0043] Please refer to Figure 2 Another embodiment of the present invention provides a system for user data transmission based on a distributed network, including a first client, a server, and a second client:

[0044] The server converts the data received from the first client into dictionary-formatted data, which includes the structural attribute features of the data and the data content features compressed by a formula, and stores the dictionary-formatted data.

[0045] In response to the download request for the data, the second client downloads the dictionary-formatted data from the server and restores the dictionary-formatted data to the original data format using a formula.

[0046] As described above, the beneficial effects of this invention are as follows: After receiving the data initially sent by the first client, the server converts it into dictionary-format data to compress and simplify the data by utilizing the shared structural features of the dictionary-format data storage. When the second client sends a download request for data, it downloads the dictionary-format data from the server and restores it to the original data format using a formula. In this way, the second client decompresses the specified data using a dictionary, eliminating the need to repeatedly download duplicate data during data transmission. Furthermore, data with the same structure share a common structure, establishing a dictionary-data storage technology to achieve data "slimming" and data dictionary format conversion. Simultaneously, by using data dictionary format conversion rather than data encryption, and without changing the transmission mode, data transmission efficiency is improved while ensuring data transmission security.

[0047] Furthermore, the server converts the received data from the first client into dictionary-formatted data. This dictionary-formatted data includes structural attribute features and data content features compressed using a formula, including:

[0048] The server obtains data from the first client, uses the data format of the data as the structural attribute feature, and uses the data other than the structural attribute feature as the data content feature;

[0049] If the data content features can be compressed using a formula, then formula-compressed data is obtained; otherwise, a data storage segment is obtained, and the formula-compressed data, the data storage segment, and the structural attribute features are used as dictionary-formatted data.

[0050] As described above, for data content features that cannot be processed by the dictionary, they are directly generated as data storage segments. For data content features that can be processed by the dictionary, corresponding formula compressed data is generated. Therefore, dictionary-formatted data includes not only structural attribute features but also data storage segments and formula compressed data in the data content features, which can handle multiple data types and further improve data transmission efficiency.

[0051] Furthermore, taking the data format of the data as a structural attribute feature, and the data other than the structural attribute feature as a data content feature, the method further includes:

[0052] The structural attribute features and data content features of the data are stored in the corresponding nodes in the form of a binary tree.

[0053] As described above, structural attribute features and data content features are stored in a binary tree to facilitate the subsequent restoration of dictionary-formatted data.

[0054] Furthermore, storing the dictionary-formatted data includes:

[0055] The compressed formula data is segmented and the segmented compressed formula data is stored in a binary tree node.

[0056] As described above, by segmenting data, a dictionary-data-segmentation storage technology can be established to further reduce data size.

[0057] Furthermore, the dictionary-formatted data is restored to its original format using a formula, including:

[0058] Based on the structural attribute features in the dictionary-formatted data, generate the structural information of the corresponding data;

[0059] The compressed formula data in the dictionary format data is restored, and the restored compressed formula data and the data storage segment are filled into the structure information of the data.

[0060] As described above, generating structural information by using structural attribute features from dictionary-formatted data and then filling it with data restored from the dictionary-formatted data can effectively improve data transmission efficiency.

[0061] The method and system for user data transmission based on a distributed network described above are suitable for improving data transmission efficiency while ensuring data transmission security. The following specific implementation methods illustrate this:

[0062] Example 1

[0063] Please refer to Figure 1 , Figure 3 and Figure 4 A method for user data transmission based on a distributed network, comprising the following steps:

[0064] S1. The server converts the data received from the first client into dictionary format data, which includes the structural attribute features of the data and the data content features compressed by a formula, and stores the dictionary format data.

[0065] Specifically, after the first client uploads data to the server, the server processes the data and converts it into a dictionary format for storage. The steps performed by the server are as follows:

[0066] S11. The server obtains the data from the first client, uses the data format of the data as the structural attribute feature, and uses the data other than the structural attribute feature as the data content feature, and stores the structural attribute feature and data content feature of the data in the corresponding nodes in the form of a binary tree.

[0067] In this embodiment, after the server obtains the data, it extracts the data according to the data structure to obtain the structural attribute features and data content features of the data, and stores both in the form of a binary tree on the corresponding nodes.

[0068] Among them, structural attribute features are those that use data format as common features. For example, each MP3 file has a common data structure in its data header, and this common data structure is used as the structural attribute feature.

[0069] Data content features are the features other than structural attribute features, and the members in the dictionary are used as the stored feature identifiers.

[0070] S12. If the data content features can be compressed using a formula, then formula-compressed data is obtained; otherwise, a data storage segment is obtained, and the formula-compressed data, the data storage segment, and the structural attribute features are used as dictionary-formatted data.

[0071] Specifically, the data content features are acquired, matched with data formulas, and then simplified based on these formulas. In this embodiment, the formula template contains various formulas. When matching data formulas, the data is first loaded into the formula template to obtain the processed data, which is then compared with the original data. If the similarity is greater than a threshold, it indicates that the processed data is still quite similar to the original data, and the data can be further compressed, allowing adjustment of the factors in the formula. If the similarity indicates that the processed data is completely identical to the original data, the processed data can be directly simplified, and the corresponding position and formula can be directly generated on the binary tree.

[0072] This also includes:

[0073] The compressed formula data is segmented and the segmented compressed formula data is stored in a binary tree node.

[0074] S2. In response to the download request for the data, the second client downloads the dictionary-formatted data from the server and restores the dictionary-formatted data to the original data format using a formula.

[0075] Specifically, based on the structural attribute features in the dictionary-formatted data, the structural information of the corresponding data is generated;

[0076] The compressed formula data in the dictionary format data is restored, and the restored compressed formula data and the data storage segment are filled into the structure information of the data.

[0077] Example 2

[0078] Please refer to Figure 2A system for user data transmission based on a distributed network includes a first client, a server, and a second client. The first client implements the steps executed by the first client as the execution subject in the method for user data transmission based on a distributed network according to Embodiment 1. The server implements the steps executed by the server as the execution subject in the method for user data transmission based on a distributed network according to Embodiment 1. The second client implements the steps executed by the second client as the execution subject in the method for user data transmission based on a distributed network according to Embodiment 1.

[0079] In summary, the present invention provides a method and system for user data transmission based on a distributed network. After receiving the data initially sent by the first client, the server converts it into dictionary-formatted data to leverage the shared structural characteristics of the dictionary-formatted data for compression and simplification. When the second client sends a download request for data, it downloads the dictionary-formatted data from the server and restores it to its original format using a formula. In this way, the second client decompresses the specified data using a dictionary, eliminating the need to repeatedly download duplicate data during transmission. Furthermore, data with the same structure share a common structure, establishing a dictionary-data storage technology to achieve data "slimming" and dictionary format conversion. Since data dictionary format conversion, rather than data encryption, does not require changing the transmission mode, it improves data transmission efficiency while ensuring data transmission security.

[0080] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for user data transmission based on a distributed network, characterized in that, The method comprises the steps of: The server converts the received data of the first client into dictionary format data, which comprises structural attribute features and formula-compressed data content features of the data, and stores the dictionary format data; The second client downloads the dictionary format data from the server in response to a download request for the data, and restores the dictionary format data to the original data format through formula; The server converts the received data of the first client into dictionary format data, which comprises structural attribute features and formula-compressed data content features of the data, comprising: The server obtains the data of the first client, takes the data format of the data as the structural attribute features, and takes the data other than the structural attribute features as the data content features; If the data content features can be compressed through formula, formula-compressed data is obtained, otherwise, a data storage segment is obtained, and the formula-compressed data, the data storage segment, and the structural attribute features are taken as the dictionary format data.

2. The method for user data transmission based on distributed network according to claim 1, characterized in that, Taking the data format of the data as the structural attribute features, and taking the data other than the structural attribute features as the data content features, further comprising: Storing the structural attribute features and the data content features of the data in corresponding nodes in the form of a binary tree.

3. The method for user data transmission based on distributed network according to claim 2, characterized in that, Storing the dictionary format data, comprising: Segmenting and cutting the formula-compressed data, and storing the cut formula-compressed data in the binary tree nodes.

4. The method for user data transmission based on distributed network according to claim 1, characterized in that, Restoring the dictionary format data to the original data format, comprising: Generating structural information of the corresponding data according to the structural attribute features in the dictionary format data; Restoring the formula-compressed data in the dictionary format data, and filling the restored formula-compressed data and the data storage segment into the structural information of the data.

5. A system for user data transmission based on a distributed network, characterized by The method comprises a first client, a server, and a second client: The server converts the received data of the first client into dictionary format data, which comprises structural attribute features and formula-compressed data content features of the data, and stores the dictionary format data; The second client downloads the dictionary format data from the server in response to a download request for the data, and restores the dictionary format data to the original data format through formula; The server converts the received data of the first client into dictionary format data, which comprises structural attribute features and formula-compressed data content features of the data, comprising: The server obtains the data of the first client, takes the data format of the data as the structural attribute features, and takes the data other than the structural attribute features as the data content features; If the data content features can be compressed through formula, formula-compressed data is obtained, otherwise, a data storage segment is obtained, and the formula-compressed data, the data storage segment, and the structural attribute features are taken as the dictionary format data.

6. The system for user data transmission based on distributed network according to claim 5, wherein, Taking the data format of the data as the structural attribute features, and taking the data other than the structural attribute features as the data content features, further comprising: The structural attribute feature and the data content feature of the data are stored in corresponding nodes in a binary tree form.

7. A system for user data transmission based on a distributed network according to claim 6, characterized in that, The dictionary format data is stored, including: The formula compression data is segmented and cut, and the cut formula compression data is stored in a binary tree node.

8. The system for user data transmission based on distributed network according to claim 5, wherein, The dictionary format data is restored to the original data format through a formula, including: According to the structural attribute feature in the dictionary format data, structural information of corresponding data is generated; The formula compression data in the dictionary format data is restored, and the restored formula compression data and the data storage segment are filled into the structural information of the data.

Images