File compression method and device, storage medium and electronic device

Abstract

The embodiment of the invention provides a file compression method and device, a storage medium and an electronic device, and the method comprises the steps: combining a plurality of files to form a compression training set, so as to cooperatively generate a data dictionary; scanning and selecting a first data segment and a second data segment from the compression training set, and respectively analyzing a first repetition frequency of the first data segment in the compression training set and a second repetition frequency of the second data segment in the compression training set; when the first number of repetitions is greater than the second number of repetitions, marking the first data segment into the data dictionary; otherwise, re-analyzing the first number of repetitions and the second number of repetitions. By means of the file compression method and device, the technical problem that in the related technology, the compression effect is poor due to the fact that the customization analysis capacity of file compression is insufficient and the compression path of the file in the specific form is not accurate enough is solved, and then the effect of customized compression of the file in the specific form is achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of computer data compression, and in particular, to a file compression method, device, storage medium and electronic device. Background Art

[0002] With the development of the Internet, especially the development of mobile Internet, the file exchange between clients and servers and between servers is becoming more and more frequent and time-consuming. At present, the transmitted files are mainly compressed using gz ip and other internationally accepted traditional compression algorithms. The compression idea is not targeted at files of a specific form and has a certain degree of universality. It does not have customized scene analysis and file intelligent analysis capabilities. Although it can solve some file compression problems and reduce the size of files, the compression capability does not achieve the best effect, especially for files of a specific form, and it cannot compress the content with a high repetition rate well, which consumes a lot of network resources and transmission time. Summary of the invention

[0003] The embodiments of the present invention provide a file compression method, device, storage medium and electronic device to at least solve the technical problems in the related art that the customized analysis capability of file compression is lacking, the compression path for files of a specific form is not accurate enough, and the compression effect is poor.

[0004] According to an embodiment of the present invention, a file compression method is provided, comprising the following steps:

[0005] Merge several files to form a compressed training set to facilitate the generation of a data dictionary;

[0006] Scan and select a first data segment and a second data segment from the compressed training set, the first data segment and the second data segment are at the same starting position of the compressed training set, the length of the first data segment is less than the length of the second data segment by a first number of bytes, and analyze a first repetition number of the first data segment in the compressed training set and a second repetition number of the second data segment in the compressed training set respectively;

[0007] When the first number of repetitions is greater than the second number of repetitions, the first data segment is marked in the data dictionary; otherwise, the second number of bytes is increased for the first data segment length and the second data segment length, and the first number of repetitions and the second number of repetitions are re-analyzed.

[0008] In an exemplary embodiment, after marking the first data segment to the data dictionary, the starting positions of the first data segment and the second data segment are also moved back by the length of the first data segment, and the first repetition number and the second repetition number are re-analyzed until the entire compressed training set is scanned.

[0009] In an exemplary embodiment, the first byte number and the second byte number are both one byte.

[0010] In an exemplary embodiment, the starting positions of the first data segment and the second data segment begin from the first byte position of the compressed training set, and the length of the first data segment is seven bytes.

[0011] In an exemplary embodiment, the first file is compressed based on the data dictionary. When it is analyzed that the compression ratio of the first file is worse than a preset threshold range, the starting positions of the first data segment and the second data segment and the length of the first data segment are adjusted, and the data dictionary is regenerated to achieve the best compression effect.

[0012] In an exemplary embodiment, the second file is compressed based on the data dictionary, and the second file is further compressed by a Fourier wavelet transform algorithm.

[0013] In an exemplary embodiment, the merging of several files is performed by classifying and merging the files according to usage scenarios of the files.

[0014] According to another embodiment of the present invention, there is provided a file compression device, comprising:

[0015] A merging module is used to merge several files to form a compressed training set to facilitate the generation of a data dictionary;

[0016] an analysis module, configured to scan and select a first data segment and a second data segment from the compressed training set, wherein the first data segment and the second data segment have the same starting position in the compressed training set, and the length of the first data segment is less than the length of the second data segment by a first number of bytes, and respectively analyze a first repetition number of the first data segment in the compressed training set and a second repetition number of the second data segment in the compressed training set;

[0017] A generation module is used to mark the first data segment in the data dictionary when the first repetition number is greater than the second repetition number; otherwise, increase the second number of bytes to the first data segment length and the second data segment length respectively, and re-analyze the first repetition number and the second repetition number.

[0018] In an exemplary embodiment, after marking the first data segment to the data dictionary, the starting positions of the first data segment and the second data segment are also moved back by the length of the first data segment, and the first repetition number and the second repetition number are re-analyzed until the entire compressed training set is scanned.

[0019] In an exemplary embodiment, a verification module is also included. The verification module compresses the first file based on the data dictionary. When analyzing that the compression ratio of the first file is worse than a preset threshold range, the starting position of the first data segment and the second data segment and the length of the first data segment are adjusted, and the data dictionary is regenerated to achieve the best compression effect.

[0020] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0021] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0022] Through the present invention, a compression training set is formed by merging several files to cooperate with the generation of a data dictionary; the first data segment and the second data segment are scanned and selected from the compression training set, the first data segment and the second data segment are at the same starting position of the compression training set, the length of the first data segment is less than the length of the second data segment by a first number of bytes, and the first repetition number of the first data segment in the compression training set and the second repetition number of the second data segment in the compression training set are analyzed respectively; when the first repetition number is greater than the second repetition number, the first data segment is marked in the data dictionary; otherwise, the second number of bytes is added to the length of the first data segment and the length of the second data segment respectively, and the first repetition number and the second repetition number are re-analyzed. Therefore, the technical problems of the lack of customized analysis capability of file compression in the related art, the inaccurate compression path of files of a specific form, and the poor compression effect can be solved, so as to achieve the effect of customized compression of files of a specific form. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a hardware structure block diagram of a computer terminal according to a file compression method of an embodiment of the present invention;

[0024] Figure 2 is a flowchart of a file compression method according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of selecting a first data segment and a second data segment according to an embodiment of the present invention;

[0026] Figure 4 is a structural block diagram of a file compression device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a computer terminal of a file compression method according to an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the file compression method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] In this embodiment, a file compression method running on the above-mentioned computer terminal or other architecture is provided. Figure 2 is a flowchart of a file compression method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0033] Step S202, merging several files to form a compressed training set to facilitate the generation of a data dictionary;

[0034] Step S204, scanning and selecting a first data segment and a second data segment from the compressed training set, the first data segment and the second data segment have the same starting position in the compressed training set, the length of the first data segment is less than the length of the second data segment by a first number of bytes, and analyzing a first repetition number of the first data segment in the compressed training set and a second repetition number of the second data segment in the compressed training set respectively;

[0035] Step S206, when the first number of repetitions is greater than the second number of repetitions, mark the first data segment in the data dictionary; otherwise, increase the second number of bytes to the length of the first data segment and the length of the second data segment, respectively, and re-analyze the first number of repetitions and the second number of repetitions.

[0036] Through the above steps, multiple sample files are merged into the same type, and the merged compression training sets with the same type of file features are uniformly analyzed. A closed-loop algorithm is used to find the character string or code (i.e., the first data segment) with a high probability of appearing in the compression training set, and the corresponding data dictionary is continuously updated, so that the data dictionary used as a compression reference sample can greatly improve the file compression effect. In this embodiment, the files are classified according to the project and the files are analyzed one by one in a scenario-based manner, so the compression ratio and effect are much better than the traditional algorithm, which is particularly suitable for the scenario of transmitting a large number of similar files.

[0037] In step S202, first, several files are merged. Preferably, the merged files need to be classified according to the usage scenarios of the files. The reason for classification according to the usage scenarios is that the traditional compression algorithm only compresses the character strings or codes that generally have a high probability of repetition, but ignores that in the usage scenarios under a specific form, there are many character strings or codes in the same type of files that are not often used in other usage scenarios but have a very high probability of repetition in this usage scenario. According to actual analysis, it is found that there are many scenarios in which the same type of files are transmitted. In these same type of files, there are many similar or even completely repeated contents. According to statistics, about 50%-80% of the contents are repeatedly transmitted. If the above-mentioned repeated contents are determined, the compression effect can be greatly improved by replacing them with relatively short reference tags. Therefore, the same type of files that need to be compressed are classified, especially the files of the same usage scenario are classified, such as the merged files are all from the same server or the same system. Then analyze them, it can reduce the impact of other usage scenario files on the confirmation of the character strings or codes with high probability of repetition in the usage scenario files. Merging several files into a compressed training set can be understood as merging several files into a similar file to facilitate overall analysis. In more embodiments, several files can also be placed in the same folder, and the files in the same folder are scanned and analyzed by the program to ensure that other different types of files are not mixed for analysis. In theory, the more files that are merged, the better, and the more obvious the scenario features, the better. In actual embodiments, all files to be compressed can be merged and analyzed according to the usage scenario, or representative files can be extracted for merged analysis. In more embodiments, the sample range of the compressed training set can also be continuously updated based on a dynamic sampling method. As the file storage in the usage scenario continues to increase, the number of merged files in the compressed training set is continuously enriched. Preferably, a corresponding range configuration interface can also be set to adjust the sample range of the compressed training set, such as judging the computing power available for analysis to adjust the sample size of the compressed training set suitable for the computing power.

[0038] After forming the compression training set in step S202, the data dictionary can be generated by scanning and analyzing the compression training set, thereby providing a corresponding basis for compression. It should be noted that the data dictionary stores character strings and codes with high repetition probability used for data compression, and these character strings and codes constitute sample data for data compression. Therefore, quickly obtaining a data dictionary with more content and a high probability of repetition of sample data becomes an important part of file compression, providing a path for accurate compression. Further in step S204, the compression training set is scanned, and the process is to continuously search for data segments with high repetition probability in the compression training set. Since the data segments with high repetition probability marked in the data dictionary do not have any obvious feature judgment basis, it is completely impossible to determine the data segments with high repetition probability by judging a specific mark. Therefore, in this embodiment, the first data segment is intercepted in the compression training set according to a specific rule, and then compared with other data contents in the compression training set, the content of the first data segment is adjusted as needed, and then compared with other data contents in the compression training set, and so on and so forth until the data segment with high repetition probability is determined.

[0039] Specifically, the selection of the first data segment is determined by the starting position and length. Preferably, the starting position of the first data segment is the starting position of the compressed training set, that is, the first byte position of the compressed training set. According to the analysis experience of the compression effect, the length of the first data segment is determined to start from seven bytes by default. Figure 3 As shown, when scanning the compressed training set, preferably the first data segment is selected from the first seven bytes of the compressed training set, and the second data segment is further selected as the reference data segment. The second data segment is very close to the first data segment in content, and the second data segment is consistent with the starting position of the first data segment. The starting position of the second data segment is also the starting position of the compressed training set. At the same time, the length of the first data segment is less than the length of the second data segment by the first number of bytes, that is, the length of the second data segment is more than the first data segment by a certain number of bytes. In this embodiment, the first number of bytes that differ is one byte. Accordingly, the length of the first data segment is seven bytes, and the length of the second data segment is eight bytes. It should be noted that the starting position and length of the first data segment and the second data segment of the initial scan can be adjusted according to the situation.

[0040] After determining the first data segment and the second data segment, the first repetition number of the first data segment in the compressed training set and the second repetition number of the second data segment in the compressed training set are analyzed respectively. This is to compare whether the probability of repetition of the first data segment in the compressed training set is higher or the probability of repetition of the second data segment in the compressed training set is higher. The determination of the number of repetitions can be performed by moving byte by byte in the compressed training set or by moving according to the length of the first data segment, and constantly comparing with different data segments in the compressed training set. Once the same, the number of repetitions is recorded until it moves to the end of the compressed training set. After determining the first number of repetitions and the second number of repetitions, step S206 can be entered to compare the size of the first number of repetitions and the second number of repetitions. When the first number of repetitions is greater than the second number of repetitions, it means that the first data segment can be determined as a data segment with a high probability of repetition, and the first data segment is directly marked in the data dictionary. When the first number of repetitions is less than the second number of repetitions, it means that the first data segment does not meet the requirements for marking in the data dictionary. At this time, it is necessary to adjust the content of the first data segment to further find a data segment with a high probability of repetition, and further increase the second number of bytes to the length of the first data segment and the second data segment, that is, further extend the length of the first data segment and the second data segment to determine a longer data segment. Preferably, the second number of bytes is also set to one byte. At this time, it can be understood that the second data segment is used as the first data segment, and the second data segment is extended by one byte to make the length of the second data segment one byte longer than the length of the first data segment. Step S204 is re-entered to analyze the first number of repetitions of the re-determined first data segment in the compressed training set and the second number of repetitions of the re-determined second data segment in the compressed training set, and then step S206 is executed, and the cycle is repeated until the re-determined first data segment is marked in the data dictionary.

[0041] Further, in any loop, as long as it is determined that the first number of repetitions is greater than the second number of repetitions, the first data segment will be marked as a high repetition probability data segment in the data dictionary, and the corresponding loop will be terminated at this time, and the length of the first data segment and the second data segment will not be further extended. At this time, there is a situation that other data segments in the compressed training set have not been scanned and analyzed. Therefore, preferably, the starting position of the first data segment and the second data segment will be moved back by the length of the first data segment. Taking the length of seven bytes as an example, if the first data segment is marked in the data dictionary, the first data segment and the second data segment will be reselected from the eighth byte. Similarly, if the first data segment marked in the data dictionary is eight bytes, the re-determined first data segment and the second data segment will start from the ninth byte. The re-determined first data segment and the second data segment will re-analyze the first number of repetitions and the second number of repetitions in step S204, and then enter step S206 to determine whether they can be marked in the data dictionary. If the first data segment and the second data segment are not satisfied, the first data segment and the second data segment will be re-adjusted, and steps S204 and S206 will be executed cyclically until the above-mentioned end condition is met. In this embodiment, it can be understood that two nested double loops are constructed. After the first data segment is marked to the data dictionary, the starting positions of the first data segment and the second data segment are moved back by the length of the first data segment to recalculate the number of repetitions in the outer loop. The condition for ending the loop is that the selected first data segment has scanned the entire compressed training set.

[0042] The data dictionary generated in the above manner can effectively cope with the compression scenario of files of a specific form, and can also adjust the customized compression requirements according to actual needs, effectively improving the compression effect of files in specific usage scenarios. Further, the updated data dictionary will be distributed to the corresponding sending end (client or server) and receiving end (client or server) for use. Preferably, the data dictionary will be encrypted and transmitted before distribution to ensure the security of use. The sending end and the receiving end will decrypt and use it after receiving the encrypted data dictionary. In order to further improve the ability of closed-loop analysis and cope with the fluctuation of the data dictionary quality caused by the file content in special circumstances, the first file will be compressed based on the data dictionary. When the compression ratio of the first file is analyzed to be worse than the preset threshold range, the starting position of the first data segment and the second data segment and the length of the first data segment are adjusted, and the data dictionary is regenerated to achieve the best compression effect. The first file can be a sampled test file, or a file that actually needs to be transmitted. The first file is compressed according to the generated data dictionary, and the compression result is analyzed. Specifically, a threshold range of compression ratio is pre-set to determine whether the compression effect is better or worse than the preset value. If it is worse than the preset threshold range, it means that the quality of the generated data dictionary is not good. At this time, the parameters of the first data segment and the second data segment can be adjusted. As mentioned above, the starting position and length of the first data segment and the second data segment can be adjusted. After adjustment, the loop of step S204 and step S206 is executed again until a new data dictionary is regenerated. In this way, the best data dictionary can be obtained through continuous attempts to achieve the best file compression effect. In a preferred embodiment, a Fourier wavelet transform algorithm is also introduced to cooperate with the generated data dictionary to realize file compression. The second file is first compressed based on the data dictionary, and the second file is further compressed by the Fourier wavelet transform algorithm. In this way, the second file to be compressed can finally achieve a better compression effect and reduce the consumption of resources and practice during transmission.

[0043] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0044] In this embodiment, a file compression device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0045] According to another embodiment of the present invention, a file compression device is provided. Figure 4 is a structural block diagram of a file compression device according to an embodiment of the present invention. Figure 4 As shown, the device comprises:

[0046] A merging module 22, used to merge several files to form a compressed training set to facilitate the generation of a data dictionary;

[0047] An analysis module 24 is configured to scan and select a first data segment and a second data segment from the compressed training set, wherein the first data segment and the second data segment have the same starting position in the compressed training set, and the length of the first data segment is less than the length of the second data segment by a first number of bytes, and respectively analyze a first repetition number of the first data segment in the compressed training set and a second repetition number of the second data segment in the compressed training set;

[0048] The generation module 26 is used to mark the first data segment in the data dictionary when the first repetition number is greater than the second repetition number; otherwise, increase the second number of bytes to the first data segment length and the second data segment length respectively, and re-analyze the first repetition number and the second repetition number.

[0049] Through the cooperation of the merging module 22, the analyzing module 24 and the generating module 26, multiple sample files are merged into the same type, and the merged compression training set with the same type of file features is uniformly analyzed. A closed-loop algorithm is used to find the character string or code (i.e., the first data segment) with a high probability of appearing in the compression training set, and the corresponding data dictionary is continuously updated, so that the data dictionary used as a compression reference sample can greatly improve the file compression effect. In this embodiment, the files are classified according to the project and the files are analyzed one by one in a scenario-based manner, so the compression ratio and effect are much better than the traditional algorithm, which is particularly suitable for the scenario of transmitting a large number of similar files.

[0050] Specifically, the analysis module 204 and the generation module 206 can scan data segments of different lengths at different positions in the compression training set by adjusting the starting position and length of the first data segment and the second data segment, and further implement cyclical scanning through repeated calls between the analysis module 204 and the generation module 206, and finally generate a data dictionary with the best quality as a reference sample for file compression.

[0051] In an exemplary embodiment, after marking the first data segment to the data dictionary, the starting positions of the first data segment and the second data segment are also moved back by the length of the first data segment, and the first repetition number and the second repetition number are re-analyzed until the entire compressed training set is scanned.

[0052] In an exemplary embodiment, the first byte number and the second byte number are both one byte.

[0053] In an exemplary embodiment, the starting positions of the first data segment and the second data segment begin from the first byte position of the compressed training set, and the length of the first data segment is seven bytes.

[0054] In an exemplary embodiment, a verification module is also included. The verification module compresses the first file based on the data dictionary. When analyzing that the compression ratio of the first file is worse than a preset threshold range, the starting position of the first data segment and the second data segment and the length of the first data segment are adjusted, and the data dictionary is regenerated to achieve the best compression effect.

[0055] In an exemplary embodiment, a compression module is further included, wherein the compression module compresses the second file based on the data dictionary and further compresses the second file through a Fourier wavelet transform algorithm.

[0056] In an exemplary embodiment, the merging module classifies and merges a plurality of files according to usage scenarios of the files.

[0057] Furthermore, the specific implementation of the file compression device may also refer to the specific implementation of the file compression method.

[0058] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0059] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0060] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0061] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0062] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0063] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0064] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0065] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A file compression method, characterized in that: The steps include: Merge several files to form a compressed training set to facilitate the generation of a data dictionary; Scan and select a first data segment and a second data segment from the compressed training set, the first data segment and the second data segment are at the same starting position of the compressed training set, the length of the first data segment is less than the length of the second data segment by a first number of bytes, and analyze a first repetition number of the first data segment in the compressed training set and a second repetition number of the second data segment in the compressed training set respectively; When the first number of repetitions is greater than the second number of repetitions, the first data segment is marked in the data dictionary; otherwise, the second number of bytes is increased for the first data segment length and the second data segment length, and the first number of repetitions and the second number of repetitions are re-analyzed.

2. The file compression method according to claim 1, characterized in that: After marking the first data segment to the data dictionary, the starting positions of the first data segment and the second data segment are also moved back by the length of the first data segment, and the first repetition number and the second repetition number are re-analyzed until the entire compressed training set is scanned.

3. The file compression method according to claim 1, characterized in that: The first byte number and the second byte number are both one byte.

4. The file compression method according to claim 1, characterized in that: The starting positions of the first data segment and the second data segment begin from the first byte position of the compressed training set, and the length of the first data segment is seven bytes.

5. The file compression method according to claim 1 or 4, characterized in that: The first file is compressed based on the data dictionary. When the compression ratio of the first file is analyzed to be worse than a preset threshold range, the starting positions of the first data segment and the second data segment and the length of the first data segment are adjusted, and the data dictionary is regenerated to achieve the best compression effect.

6. The file compression method according to claim 1, characterized in that: The second file is compressed based on the data dictionary, and the second file is further compressed by a Fourier wavelet transform algorithm.

7. The file compression method according to claim 1, characterized in that: The merging of several files is classified and merged according to the usage scenarios of the files.

8. A file compression device, characterized in that: include: A merging module is used to merge several files to form a compressed training set to facilitate the generation of a data dictionary; an analysis module, configured to scan and select a first data segment and a second data segment from the compressed training set, wherein the first data segment and the second data segment have the same starting position in the compressed training set, and the length of the first data segment is less than the length of the second data segment by a first number of bytes, and respectively analyze a first repetition number of the first data segment in the compressed training set and a second repetition number of the second data segment in the compressed training set; A generation module is used to mark the first data segment in the data dictionary when the first repetition number is greater than the second repetition number; otherwise, increase the second number of bytes to the first data segment length and the second data segment length respectively, and re-analyze the first repetition number and the second repetition number.

9. The file compression device according to claim 8, characterized in that: After marking the first data segment to the data dictionary, the starting positions of the first data segment and the second data segment are also moved back by the length of the first data segment, and the first repetition number and the second repetition number are re-analyzed until the entire compressed training set is scanned.

10. The file compression device according to claim 8, characterized in that: It also includes a verification module, which compresses the first file based on the data dictionary. When analyzing that the compression ratio of the first file is worse than a preset threshold range, the starting position of the first data segment and the second data segment and the length of the first data segment are adjusted, and the data dictionary is regenerated to achieve the best compression effect.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images