A cloud-based dictionary lossless compression method

Abstract

This invention discloses a byte-based lossless compression method based on a cloud dictionary. The method involves establishing a cloud dictionary and accessing it online; calculating the header fields of the compressed file or process file based on the source file; performing probabilistic statistics on the source file bytes by bytes, generating compressed encoded content using the Huffman algorithm, and using this as an index to obtain the compression encoding level and number of the corresponding tuple in the cloud dictionary's compressed encoded content field; compressing the source file into a binary file according to the correspondence, padding it to integer bytes, and saving it as a process file; comparing the length of the compressed file with the previous process file, and outputting the final compressed file when the compression ratio meets the requirements. By using a remote cloud dictionary, local dictionary storage space is saved, significantly reducing the header length of the compressed file and process file, making multiple compressions feasible. Through multiple compressions, high-compression-ratio lossless compression can be achieved for large datasets.

Description

Technical Field

[0001] This invention relates to a compression method, and more particularly to a lossless compression method based on bytes and relying on a cloud dictionary. Background Technology

[0002] Data compression is divided into lossy compression and lossless compression. Lossless compression can be further divided into statistical model-based compression methods and dictionary-based compression methods. Statistical model-based compression methods inevitably generate a correspondence table between source and target codes for each compression. The header of the compressed file must include the necessary basic information and the correspondence table between source and target codes, which limits the possibility of repeatedly compressing the compressed file to reduce its length. Summary of the Invention

[0003] Purpose of the invention: To address the shortcomings of the existing technology, this invention provides a lossless compression method based on bytes using a cloud dictionary, which reduces header file length, performs multiple compressions on very large files, and improves compression efficiency.

[0004] Technical solution: The lossless compression method based on bytes and relying on cloud dictionaries of the present invention includes the following steps:

[0005] S1: Establish a cloud dictionary;

[0006] S2: The user submits the source file and defines the name of the final compressed file;

[0007] S3: Connect to the network to obtain the current cloud dictionary and calculate the header fields of the compressed file or process file.

[0008] S4: Perform probability statistics on the source file in bytes and generate compressed encoded content using the Huffman algorithm;

[0009] S5: Using the compressed encoded content as an index, search for the same value in the compressed encoded content field of the cloud dictionary to obtain the compression encoding level and compression encoding number of the tuple corresponding to the compressed encoded content.

[0010] S6: Compress the source file according to the corresponding relationship in the compression code number content, replace the source file with a binary compressed file, save it byte by byte, and pad it with binary "0" to form an integer byte, forming a process file;

[0011] S7: Compare the length of the compressed file with the length of the file from the previous process. If the length of the generated compressed file is greater than or equal to that of the file from the previous process, name the last process file as the final compressed file and output it. If the length of the compressed file does not meet the requirements, repeat S5 and S6 until the compression ratio meets the requirements.

[0012] The specific steps for creating a cloud dictionary are as follows:

[0013] Construct a sample dataset, use the sample data in the sample dataset as the source file to be compressed, perform probability statistics on a byte-by-byte basis using the Huffman algorithm, generate compressed encoded content, and update the compressed encoded content field;

[0014] S11: Construct an empty table in the cloud dictionary database using four fields: compression encoding level, compression encoding number, compression encoding content, and number of uses.

[0015] S12: Construct a sample dataset. Using the sample data in the sample dataset as the source file to be compressed, perform probability statistics on a byte-by-byte basis using the Huffman algorithm to generate compressed encoded content and update the compressed encoded content field. If there are identical values ​​in the compressed encoded content field of the cloud dictionary table, increment the value of the usage count field of the tuple by 1. If there are no identical values ​​in the compressed encoded content field of the cloud dictionary table, add a new tuple to the table, add the generated compressed encoded content to the compressed encoded content field of the new tuple, and add the 400-byte compressed encoded number to the compressed encoded number field of the new tuple in sequence. The value of the compressed encoded level field in the new tuple is 80, and the value of the usage count field in the new tuple is 1.

[0016] S13: Repeat step S12 for all sample data in the sample dataset to enrich the tables in the cloud dictionary;

[0017] S14: Sort the cloud dictionary table in descending order based on the usage count field. Reset the compression code level field in the cloud dictionary table according to the correspondence between the compression code number field of the compressed file and the compression code number field of the cloud dictionary, completing the initial version of the cloud dictionary. Subsequently, as users use the cloud dictionary, new versions of the cloud dictionary are created based on changes in the usage count field values ​​of each tuple. The 2 tuples with the largest usage count field values ​​are then selected. 39 The compression encoding level field of each tuple is set to 1, and the value of these 2 tuples is also set to 1. 39 The value of the compression code number field of each tuple is set sequentially to a 5-byte hexadecimal number from "0000000000" to "7FFFFFFFFF" in descending order of usage frequency. This process is repeated for tuples with compression code level fields from 2 to 79, also in descending order of usage frequency. This completes the creation of version 1 of the cloud dictionary. Subsequently, as users use the cloud dictionary, new versions of the cloud dictionary will be created based on the changes in the usage frequency field values ​​of each tuple.

[0018] The compressed file header format includes a version number field, a filename length field, a filename field, a file length field, and a compression code ID field. These five fields are represented by their English names: Version, File Name Length, FileName, File Length, and Code ID. The following fields are abbreviated as follows: Version number is represented as V, filename length as FNL, filename as FN, file length as FL, and compression code ID as CID. The number of bytes occupied by each field is shown below:

[0019]

[0020] Among them, the V field is used to store the version number of the cloud dictionary, which occupies 1 byte and is represented by an unsigned binary number. Its valid value range is 1 to 255. When the value is 0, it indicates an error.

[0021] The FNL field stores the length of the source filename in the header of the compressed file. It occupies 1 byte and is represented by an unsigned binary number. Its valid value range is 0 to 255. When the value is 0, it means that the FN field stores the number of bytes occupied by the FN field and the decimal number of the process file. When the value in the FNL field is 1 to 255, it means that the FN field stores the original filename. In this case, the value in the FNL field represents the number of bytes occupied by the FN field.

[0022] The FN field stores the source file name or process file name before this compression, occupying 1 to 255 bytes. If the file being compressed is a source file, the source file name is stored in the FN field in machine code format. If the file being compressed is a process file, the FN field stores the number of bytes occupied by the FN field and the decimal number of the process file. If the first bit of the first byte of the FN field is "0", it means that the FN field occupies 1 byte. The following 7 bits are unsigned binary numbers representing the process file that has completed compressions from 0 to 127. Here, the file after one compression of the source file is defined as the process file that has completed 0 compressions. The compression count increases by 1 for each compression of the process file. If the first two bits of the first byte of the FN field are "10", it means that the FN field occupies 2 bytes. The following 6 bits of the first byte and the 8 bits of the second byte, totaling 14 unsigned binary bits, indicate that the file being compressed is the process file that has completed compressions from 128 to 255. 14 -1 compression process file; if the first 3 bits of the first byte in the FN field are "110", it means that the FN field occupies 3 bytes. Then the last 5 bits of the first byte and the 16 bits of the second and third bytes, totaling 21 bits of unsigned binary value, indicate that the second compression has been completed. 14 Second to 2 21-1 compression process file; if the first 4 bits of the first byte in the FN field are "1110", it means that the FN field occupies 4 bytes. Then the last 4 bits of the first byte and the 24 bits of the second to fourth bytes, totaling 28 bits of unsigned binary value, indicate that the second compression has been completed. 21 Second to 2 28 -1 compression process file; if the first 4 bits of the first byte in the FN field are "1111", it means that the FN field occupies 5 bytes. Then the last 4 bits of the first byte and the 32 bits of the second to fifth bytes, totaling 36 bits of unsigned binary value, indicate that the second compression has been completed. 28 Second to 2 36 - The file is compressed once; if the number of compressions exceeds 2... 36 This can be extended to a maximum of 2 times using a similar method. 56 Second-rate.

[0023] The FL field stores the number of bytes occupied by the FL field and the file length before compression, occupying 1 to 8 bytes. This file length can be the length of the source file or the length of the process file. The file length is represented by an unsigned binary value in bytes. The first binary "0" in the first byte of the FL field appears at the Kth bit, indicating that the FL field occupies K bytes. From the first bit after the binary "0" to the last bit of the Kth byte of the FL field, the length of the file before compression is represented by an unsigned binary value. That is, if the first bit of the first byte of the FL field is "0", it means that the FL field occupies 1 byte. The last 7 bits of the first byte of the FL field represent the file length before compression using unsigned binary values. Length; if the first two bits of the first byte in the FN field are "10", it means the FN field occupies 2 bytes. The last 6 bits of the first byte and the 8 bits of the second byte, totaling 14 bits, represent the length of the original file before compression using unsigned binary values. If the first three bits of the first byte in the FN field are "110", it means the FN field occupies 3 bytes. The last 5 bits of the first byte, the 8 bits of the second byte, and the 8 bits of the third byte, totaling 21 bits, represent the length of the original file before compression using unsigned binary values. This continues until the first 8 bits of the first byte in the FN field are "11111110", indicating the FN field occupies 8 bytes, at which point the supported file length is 2 bytes. 56 byte.

[0024] The CID field stores the compression code level (abbreviated as CL) in the cloud dictionary for this compression call, along with the compression code number CN in the cloud dictionary. It occupies 5-400 bytes. The value of CL is the position of the first binary "0" in the CID field, and the number of bytes occupied by the CID field is CL value × 5. The value of CN is the unsigned binary value consisting of all bits after the first binary "0" in the CID field. If the first binary "0" in the CID field appears in the first bit of the first byte, meaning the compression code used in this call is at level 1 in the cloud dictionary, the CID field occupies 5 bytes (40 bits). The bit describing CL in the cloud dictionary is the first bit in the CID field, and the bits describing CN in the cloud dictionary are bits 2-40 (39 bits in total). Therefore, this compression uses level 1 (2 bits) in the cloud dictionary. 39 One of the compression codes; if the first two bits of the first byte in the CID field are "10", that is, CL is level 2 in the cloud dictionary, it means that the CID field occupies 10 bytes, a total of 80 bits. Among them, the bits describing CL in the cloud dictionary are the first two bits of the CID field, and the bits describing CN in the cloud dictionary are the 3rd to 80th bits, a total of 78 bits. Then, this compression uses level 2, which is the most frequently used code in the cloud dictionary. 78 One of the compression codes; and so on. The specific correspondence between the compressed file and the compression code number field of the cloud dictionary is shown in Table 1:

[0025] Table 1. CID Field Correspondence Table

[0026]

[0027]

[0028] The cloud dictionary is a database stored in the cloud that describes the compression codes used during the compression process, along with their levels, numbers, and usage counts. Each version in the compressed file header format version number corresponds to a cloud dictionary. Each cloud dictionary contains four fields: compression code level, compression code number, compression code content, and usage count. Their data types are shown in the table below:

[0029] The data types of the fields included in Table 2 of the cloud dictionary

[0030] Compression encoding level Compression code number Compressed content Number of times used Unsigned integer Variant type Variant type Unsigned long integer

[0031] Among them, the compression encoding level and the CL in the CID field of the compressed file header format are the same concept. Its value range is an unsigned integer from 1 to 80. The smaller the value, the higher the level, indicating that the encoding is used more often.

[0032] The compression code number and the CID in the CID field of the compressed file header are the same concept. When initially constructing the cloud dictionary, all the correspondences between the original codes and the compressed codes can be encoded into a 3120-bit compressed code number. By compressing a large amount of data, the most frequently used 2 39 This correspondence is used to set the compression coding level to 1, renumber the data using 39 bits of binary data, generate new compression coding numbers, and then use the correspondence a maximum of 2 times. 39 +1~2 78 The encoding level is set to 2, and a new compressed encoding number is generated using 78-bit binary. Then, referring to the CID field correspondence table, the correspondence between encoding levels 3 to 79 is renumbered in the same way to generate a new compressed encoding number.

[0033] The compressed encoding content is used to store the correspondence between some or all of the 256 original codes and the compressed and transformed codes. Its format is that each unit is divided into three parts: original code, compressed and transformed code length, and compressed and transformed code. The units are arranged in ascending order of the original codes, as shown in the table below:

[0034] Table 3 Compression Encoding Format

[0035] Original code 1 The length of the encoded 1 after compression transformation Encoding 1 after compression Original encoding 2 The length of the encoded code after compression transformation is 2. Encoding after compression transformation 2 …… …… …… Original encoding n The length of the encoded n after compression transformation Encoding after compression transformation n

[0036] The original encoding consists of 8-bit binary numbers occupying 1 byte. The length of the compressed encoding is the actual length of the binary number after compression and transformation, represented as an unsigned binary number, occupying 32 bytes. The compressed encoding is generated according to the Huffman algorithm and is a variable-length binary number. Any part that is less than a whole byte is padded with binary 0s, occupying 32 bytes.

[0037] The usage count is used to record the number of times the compressed code content in each compressed code tuple in the cloud dictionary is used. In the future, new version numbers can be generated based on this. Its initial value is 0. Each time a compressed code tuple in the cloud dictionary is called using this method to compress a file, the value of the usage count field of that tuple is incremented by 1.

[0038] The steps for compressing a file based on the aforementioned cloud dictionary and compressed file header format are as follows:

[0039] Step 1: Compression uses the Huffman algorithm, treating each byte as an unsigned binary number with a value between 0 and 255. Based on the frequency of each byte's value in the source file, it is replaced with binary codes of different lengths. The corresponding value VC is found in the compressed encoding content field of the cloud dictionary according to the correspondence between the original encoding and the compressed encoding of each byte. The value VL in the compression encoding level field and the value VID in the compression encoding number field of the tuple containing VC are obtained, and the value VN in the usage count field of the tuple containing VC is incremented by 1.

[0040] Step 2: Obtain the version number, source file name length, source file name, and source file length from the cloud dictionary and use them as the values ​​of the V, FNL, FN, and FL fields in the header of the file to complete the 0th compression. Use the VID value as the value of the CID field in the header of the file to complete the 0th compression. According to the correspondence between the original encoding and the compressed encoding provided by the value of the compressed encoding content segment VC in the tuple with compression encoding number VID in the cloud dictionary, replace the source file with a binary compressed file. Save the binary compressed file byte by byte in groups of 8 bits. Finally, if there is a part that is less than 8 bits, pad it with binary "0" to make up 8 bits. This constitutes the data part of the 0th compression process file. The header and data part of the 0th compression process file together constitute the 0th compression process file.

[0041] Step 3: If this process file is the nth compressed process file, the Huffman algorithm is used. Each byte is treated as an unsigned binary number with a value between 0 and 255. Based on the frequency of each byte value in this process file, different lengths of binary encoding are used for substitution. The corresponding value VC is found in the cloud dictionary's compressed encoding content field according to the correspondence between the original encoding and the compressed encoding of each byte. The compression encoding level field VL and the compression encoding number VID in the tuple containing VC are obtained. The usage count field VN in the tuple containing VC is incremented by 1. The version number of the cloud dictionary is used as the value of the V field in the header of the (n+1)th compressed process file, and "0" is used as the value of the FNL field in the header of the (n+1)th compressed process file. The FN field is then processed according to the format of the compressed file header. The process file is configured such that n+1 is used as the value of the FN field in the header of the (n+1)th compression process file. According to the format of the FL field in the compressed file header, the length of the nth compression process file is used as the value of the FL field in the header of the (n+1)th compression process file. VID is used as the value of the CID field in the header of the (n+1)th compression process file. Based on the correspondence between the original encoding and the compressed encoding provided by the value of the compressed encoding content segment in the tuple with compression encoding number VID in the cloud dictionary, the current process file is replaced with a binary compressed file. The binary compressed file is saved byte by byte in groups of 8 bits. If there is a part that is less than 8 bits, it is padded with binary "0" to make up to 8 bits, which constitutes the data part of the (n+1)th compression process file. The header and data part of the (n+1)th compression process file together constitute the (n+1)th compression process file.

[0042] Step 4: If the length of the compressed file in step n+1 is less than the length of the compressed file in step n, then use the compressed file in step n+1 as the current compressed file and repeat steps 3 and 4. If the length of the compressed file in step n+1 is greater than or equal to the length of the compressed file in step n, proceed to step 5.

[0043] Step 5: Rename the nth process file to the user-defined final compressed file name to complete the compression.

[0044] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: It uses a remote cloud dictionary, which saves local dictionary storage space, can greatly reduce the header length of compressed files and process files, making multiple compressions feasible, and after multiple compressions, it can achieve lossless compression with a high compression ratio for big data. Detailed Implementation

[0045] The technical solution of the present invention will be further described below with reference to the embodiments.

[0046] First, establish a cloud dictionary:

[0047] S11: Construct an empty table in the cloud dictionary database using four fields: compression encoding level, compression encoding number, compression encoding content, and number of uses.

[0048] S12: If a sample data uses only three characters, namely the hexadecimal numbers "00", "77", and "FF", where "00" is used 40% of the time, "77" is used 35% of the time, and "FF" is used 25% of the time, according to the Huffman algorithm, the original code 1 "00" corresponds to a 1-bit binary "0" after compression transformation, the original code 2 "77" corresponds to a 2-bit binary "10" after compression transformation, and the original code 3 "FF" corresponds to a 2-bit binary "11" after compression transformation. The initial hexadecimal representation of the corresponding compressed code content is as follows:

[0049] Table 4 Hexadecimal representation of the initial values ​​of the sample data compression encoding

[0050]

[0051] Therefore, the data to be stored in the compressed encoded content field should be the hexadecimal number "000000000000000000000000000000000000000000000000000000000000000000000010 ... 077000000000000000000000000000000000000000000000000000000000000010800000000 00000000000000000000000000000000000000000000000000000FF0000000000000000000 If the compressed encoding content field already contains this hexadecimal data, the value of the usage count field of the tuple containing this hexadecimal data will be incremented by 1. If the compressed encoding content field does not contain this hexadecimal data, a new tuple will be created, and the hexadecimal data will be filled into the compressed encoding content field of the new tuple. The usage count field of the new tuple will be set to 1, and the compression encoding level field of the new tuple will be set to 80. If the new tuple is the 0th record in the database (0 is the starting value), the value of the compressed encoding number field of the new tuple will be set to an unsigned hexadecimal number represented by 400 bytes, and its value will be 0.

[0052] S13: Repeat step S12 for all sample data in the sample dataset to enrich the tables in the cloud dictionary;

[0053] S14: Sort the cloud dictionary table in descending order based on the usage count field. Based on the content in the table corresponding to the CID field, reset the value of the compression encoding level field in the cloud dictionary table, and select the 2 values ​​with the highest usage count field. 39 The compression encoding level field of each tuple is set to 1, and the value of these 2 tuples is also set to 1. 39The values ​​of the compression code number field of each tuple are arranged in descending order of usage frequency and set to 5-byte hexadecimal numbers from "0000000000" to "7FFFFFFFFF". For example, if the usage frequency field of the tuple containing the original compressed code content with codes "00", "77", or "FF" is the second largest value in the usage frequency field, then the compression code level field of that tuple is reset to 1, and the compression code number is 1 represented by a 5-byte hexadecimal value starting with 0, i.e., the compression code number is "0000000001". And so on, resetting the values ​​of the compression code level field and the compression code number field in tuples with compression code level fields from 2 to 79 in descending order of usage frequency. This completes the establishment of cloud dictionary version 1. Subsequently, as users use the cloud dictionary, new versions of the cloud dictionary are established based on the changes in the usage frequency field values ​​of each tuple.

[0054] The user submits a source file named Source, and defines the final compressed file name as End;

[0055] The system connects to the internet to obtain the current cloud dictionary and calculates the header of the compressed file or process file. For example, if the version number is 2, the value of the V field in the header of the compressed file is the hexadecimal value "02".

[0056] The source file is named "Source" and occupies 6 bytes. The FNL field in the header of the compressed file has the hexadecimal value "06".

[0057] The file name is in English characters and is represented by ASCII code in the FN field. The value of the FN field is represented in hexadecimal as "536F75726365".

[0058] If the source file's length is, for example, 32TB, then a file length of 32TB is 2. 45 B, represented in hexadecimal as 200000000000, requires 5 bytes plus 5 bits to represent. Adding the bits representing the length of this field in the FNL field, a total of 7 bytes are needed. Therefore, the first 7 bits of the binary string representing the number of bytes occupied by the FL field in the first byte of the file length field are "1111110", and the value of the FNL field is represented in hexadecimal as "FC200000000000".

[0059] The source file (Source) is analyzed byte by byte using probability statistics. The Huffman algorithm is then used to generate the software's compressed encoding content. Using this compressed encoding content as an index, the software searches for matching values ​​in the cloud dictionary's compressed encoding content field to obtain the compression level and compression code number of the corresponding tuple. If the compression level is "1" and the compression code number is "0000000001", the CID field is set to the hexadecimal value "0000000001". This results in the header of the compressed file or process file, totaling 20 bytes, as shown below:

[0060] Version number File name length file name File length Compression code number 02 06 536F75726365 FC200000000000 0000000001

[0061] After the header, the source file is compressed according to the corresponding relationship in the compressed encoding content with the compression encoding number "0000000001", and finally padded with binary "0" to become integer bytes;

[0062] If the compressed file becomes 1TB after one compression, and the file size can be reduced further after another compression, then the compressed file becomes a process file with the number 0. If the compressed encoded content generated by the process file after another compression has a compression encoding level of "2" in the cloud dictionary and a compression encoding number of "10000011010100001110", then the calculation method for each field of the header of the process file of the 0th compression is as follows:

[0063] The version number is 2, and the value of the V field in the header of the compressed file is the hexadecimal value "02";

[0064] The file is a procedure file, and the FNL field value is the hexadecimal value "00";

[0065] The file is the process file of the 0th compression. The FN field occupies 1 byte, with the first digit being "0" and the last 7 digits being "0000000". Therefore, the hexadecimal value of the FN field is "00".

[0066] The file length in the 0th compression process is 1T, and the value of the FL field is the hexadecimal value "FC100000000000".

[0067] The compression code number is "10000011010100001110". The CID field value is the hexadecimal value "10000011010100001110". The header of the compressed file or process file is 20 bytes long, as shown below:

[0068] Version number File name length file name File length Compression code number 02 00 00 FC100000000000 10000011010100001110

[0069] After the header, the 0th compression process file is compressed according to the corresponding relationship in the compression code content with compression code number "0000000001", and finally padded with binary "0" to become integer bytes;

[0070] Repeat the above steps to compress the file until the length of the generated compressed file is greater than or equal to that of the previous compressed file. Name the last compressed file "End" to complete the compression.

Claims

1. A lossless compression method based on bytes and relying on a cloud dictionary, characterized in that, The lossless compression method includes the following steps: S1: Establish a cloud dictionary; S2: The user submits the source file and defines the name of the final compressed file; S3: Connect to the network to obtain the current cloud dictionary and calculate the header fields of the compressed file or process file. S4: Perform probability statistics on the source file in bytes and generate compressed encoded content using the Huffman algorithm; S5: Using the compressed encoded content as an index, search for the same value in the compressed encoded content field of the cloud dictionary to obtain the compression encoding level and compression encoding number of the tuple corresponding to the compressed encoded content. S6: Compress the source file according to the corresponding relationship in the compression code number content, replace the source file with a binary compressed file, save it byte by byte, and pad it with binary "0" to form an integer byte, forming a process file; S7: Compare the length of the compressed file with the length of the file from the previous process. If the length of the generated compressed file is greater than or equal to that of the previous process file, name the last process file as the final compressed file and output it. If the length of the compressed file does not meet the requirements, repeat S5 and S6 until the compression ratio meets the requirements. If the file to be compressed is the source file, then step S6 includes: obtaining the version number of the cloud dictionary, the length of the source file name, the source file name, and the length of the source file as the values ​​of the V field, FNL field, FN field, and FL field in the header of the process file to complete the 0th compression, respectively; using the value of VID as the value of the CID field in the header of the process file to complete the 0th compression; replacing the source file with a binary compressed file according to the correspondence between the original encoding and the compressed encoding provided by the value of the compressed encoding content segment VC in the tuple with compression encoding number VID in the cloud dictionary; saving the binary compressed file byte by byte in groups of 8 bits; and finally, if there is a part that is less than 8 bits, padding it with binary "0" to make 8 bits to form the data part of the process file of the 0th compression. The header and data part of the process file of the 0th compression together constitute the process file of the 0th compression. If the file being compressed is the file that has completed the nth compression process, then step S6 includes: treating each byte as a binary unsigned number with a value between 0 and 255; replacing each byte value with a binary code of different lengths based on its frequency of occurrence; finding the same value VC in the cloud dictionary's compressed encoding content field according to the correspondence between the original encoding and the compressed encoding of each byte; obtaining the value VL in the compression encoding level field and the value VID in the compression encoding number field of the tuple containing VC; incrementing the value VN in the usage count field of the tuple containing VC by 1; using the version number of the cloud dictionary as the value of the V field in the header of the file that has completed the (n+1)th compression process, and "0" as the value of the FNL field in the header of the file that has completed the (n+1)th compression process; and according to the format of the FN field in the compressed file header, setting the nth... +1 is used as the value of the FN field in the header of the process file for the (n+1)th compression. According to the format of the FL field in the compressed file header, the length of the process file for the nth compression is used as the value of the FL field in the header of the process file for the (n+1)th compression. VID is used as the value of the CID field in the header of the process file for the (n+1)th compression. According to the correspondence between the original encoding and the compressed encoding provided by the value of the compressed encoding content segment in the tuple with compression encoding number VID in the cloud dictionary, the current process file is replaced with a binary compressed file. The binary compressed file is saved byte by byte in groups of 8 bits. If there is a part that is less than 8 bits, it is padded with binary "0" to make up to 8 bits, which constitutes the data part of the process file for the (n+1)th compression. The header and data part of the process file for the (n+1)th compression together constitute the process file for the (n+1)th compression.

2. The lossless compression method based on bytes and relying on a cloud dictionary according to claim 1, characterized in that, The cloud dictionary is a database stored in the cloud that describes the compression codes used in the compression process, as well as their levels, numbers, and usage counts. Each cloud dictionary contains four fields: compression code level (VL), compression code number (VID), compression code content (VC), and usage count (VN).

3. The lossless compression method based on bytes and relying on a cloud dictionary according to claim 1, characterized in that, Step S1 includes the following steps: S11: Construct an empty table in the cloud dictionary database using four fields: compression encoding level, compression encoding number, compression encoding content, and number of uses. S12: Construct a sample dataset, using the sample data in the sample dataset as the source file to be compressed, and perform probability statistics on a byte-by-byte basis using the Huffman algorithm to generate compressed encoded content and update the compressed encoded content field. S13: Repeat step S12 for all sample data in the sample dataset to enrich the tables in the cloud dictionary; S14: Sort the cloud dictionary table in descending order according to the usage count field. Based on the correspondence between the compression code number field of the compressed file and the compression code number field of the cloud dictionary, reset the value of the compression code level field in the cloud dictionary table to complete the initial version of the cloud dictionary. Subsequently, based on the changes in the usage count field values ​​of each tuple in the cloud dictionary during use, create a new version of the cloud dictionary.

4. The lossless compression method based on bytes and relying on a cloud dictionary according to claim 3, characterized in that, The step S12 of updating the compressed encoding content field includes: if there is no identical value in the compressed encoding content field of the cloud dictionary table, a new tuple is added to the table, the generated compressed encoding content is added to the compressed encoding content field of the new tuple, and the compressed encoding number is added to the compressed encoding number field of the new tuple in sequence. If there is an identical value in the compressed encoding content field of the cloud dictionary table, the value of the usage count field of the tuple is incremented by 1.

5. The lossless compression method based on bytes and relying on a cloud dictionary according to claim 3, characterized in that, Step S12 includes: selecting the 2 with the largest number of uses in the field. 39 The compression encoding level field of each tuple is set to 1, and the value of these 2 tuples is also set to 1. 39 The value of the compression code number field of each tuple is set to a 5-byte hexadecimal number from "0000000000" to "7FFFFFFFFF" in descending order of usage frequency. Similarly, the values ​​of the compression code level field and the compression code number field in tuples with compression code level fields of 2 to 79 are reset in descending order of usage frequency.

6. The lossless compression method based on bytes and relying on a cloud dictionary according to claim 1, characterized in that, The header fields of the compressed file or process file in step S3 include version number field V, filename length field FNL, filename field FN, file length field FL, and compression code number field CID. The version number field stores the version number of the cloud dictionary. Each version number corresponds to one cloud dictionary, occupies 1 byte, and is represented by an unsigned binary number. Its valid value range is 1 to 255. When the value is 0, it indicates an error. The filename length field stores the length of the source filename in the header of the compressed file. It occupies 1 byte and is represented by an unsigned binary number. Its valid value range is 0 to 255. When the value is 0, it means that the FN segment stores the number of bytes occupied by the FN segment and the decimal number of the process file. When the value in the FNL field is 1 to 255, it means that the FN field stores the original filename. In this case, the value in the FNL field represents the number of bytes occupied by the FN segment. The FN field stores the source file name or process file name before this compression, occupying 1 to 255 bytes. If the file being compressed is a source file, the source file name is stored in the FN field in machine code format. If the file being compressed is a process file, the FN field stores the number of bytes occupied by the FN field and the decimal number of the process file. If the first bit of the first byte in the FN field is "0", it means that the FN field occupies 1 byte. The following 7 bits represent the process file that has completed 0 to 127 compressions in unsigned binary. Here, the file after one compression of the source file is defined as the process file that has completed 0 compressions. The compression count is incremented by 1 for each compression of the process file. The FL field stores the number of bytes occupied by the FL field and the file length before this compression, occupying 1 to 8 bytes. This file length is the length of the source file or the length of the process file. The file length is represented by an unsigned binary number in bytes. The first binary "0" in the first byte of the FL field appears at the Kth bit, indicating that the number of bytes occupied by the FL field is K bytes. From the first bit after the binary "0" to the last bit of the Kth byte of the FL field, the file length before this compression is represented by an unsigned binary number. The CID field stores the compression code used in this compression call, which is the compression code level in the cloud dictionary, abbreviated as CL, and the compression code number CN in the cloud dictionary. It occupies 5 to 400 bytes. The value of CL is the position of the first binary "0" in the CID field. The number of bytes occupied by the CID field is the value of CL × 5. The value of CN is the unsigned binary value composed of all bits after the first binary "0" in the CID field.

7. The cloud-based system according to claim 6 A dictionary-based lossless compression method, characterized in that, Step S5 includes: treating each byte as a binary unsigned number with a value between 0 and 255; replacing each byte value with a binary code of different lengths based on the frequency of occurrence of each byte value in the source file; finding the same value VC in the compressed encoding content field of the cloud dictionary according to the correspondence between the original encoding and the compressed encoding of each byte; obtaining the value VL of the compression encoding level field and the value VID of the compression encoding number in the tuple containing VC; and incrementing the value VN of the usage count field in the tuple containing VC by 1.

Images