Method, device and equipment for improving compression efficiency of a compression card
By adaptively adjusting the lengths of the search buffer and the pre-read buffer based on the maximum length of the repeated strings in the data to be compressed, the problem of low compression efficiency caused by the fixed buffer is solved, and more efficient data compression is achieved.
Patent Information
- Application Number
- CN202310251046.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-10
- Publication Date
- 2026-01-23
- Estimated Expiration
- 2043-03-10
AI Technical Summary
In existing technologies, the buffer length of compression cards is fixed and cannot be adjusted according to different raw data, resulting in poor compression efficiency.
By setting adaptive search and read buffer sizes, the buffer length is dynamically adjusted based on the maximum length of repeating strings in the data to be compressed, in order to achieve the optimal compression ratio.
It improves the compression efficiency of the compression card, reduces the memory usage of the storage system, and enhances the data compression effect.
Smart Images

Figure CN116185303B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, and device for improving the compression efficiency of compression cards. Background Technology
[0002] In the storage field, compressed volumes are widely used to reduce data size. However, compression consumes significant system resources, such as CPU and memory. To address this issue, an auxiliary device called a compression card has emerged. A compression card is an external auxiliary processing device specifically designed for compressing data. It connects to the storage server via a PCIe (Peripheral Component Interconnect Express) bus. Data to be compressed enters the CPU of the compression card directly, and the compressed data is then sent to the storage server.
[0003] The smaller the data size after compression using a compression card, the better, as this saves more hard drive resources and reduces user costs. Current compression technologies use a fixed buffer (WindowSize) to compress data. WindowSize includes a search buffer and a look-ahead buffer, both of which are fixed values. The search buffer stores data that has already been encoded, while the look-ahead buffer stores data that has not been encoded. Specifically, the original data is first read into the look-ahead buffer, then retrieved and compared with data in the search buffer to check for similarities. If similarities are found, the original data is compressed. The more identical data found, the higher the compression ratio and the better the compression effect.
[0004] The problem with this compression method in the existing technology is that the lengths of the two buffers of WindowSize are fixed values. This means that when processing different raw data, the size of WindowSize cannot be changed randomly, and the compression process cannot be adjusted accordingly for specific raw data, which is not conducive to improving compression efficiency. Summary of the Invention
[0005] In view of this, the present invention proposes a method, apparatus, and device for improving the compression efficiency of a compression card. The method involves setting a search buffer and a pre-read buffer size adapted to the original data to be compressed, and setting these sizes with reference to the maximum length of repeated strings in the original data. Since the maximum length of repeated strings varies among different original data sets, the WindowSize is changed from a fixed value to an adaptive value derived from the original data, thereby achieving the optimal compression ratio.
[0006] Based on the above objectives, one aspect of the present invention provides a method for improving the compression efficiency of a compression card, the method comprising the following steps: determining the length value of the original data segment based on a predefined number of segmentations for the original data segmentation and a predefined length value of the search buffer; updating the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment, to obtain an updated length value of the search buffer corresponding to the current segment; compressing the original data from the beginning position of the current segment according to the updated length value of the search buffer; and performing the next segmentation and compression according to the segment length value from the end position where the current segment has completed compression, repeating until the original data has been compressed.
[0007] In some embodiments, the method further includes: updating the length value of a pre-read buffer used to store uncompressed data according to the length value of the search buffer of the current segment, so as to obtain the updated length value of the pre-read buffer corresponding to the current segment.
[0008] In some embodiments, compressing the original data from the starting position of the current segment according to the updated length value of the search buffer includes:
[0009] The original data is compressed based on the update length value of the search buffer corresponding to the current segment, the update length value of the pre-read buffer, and the starting position of the previous segment.
[0010] In some embodiments, determining the length value of the original data segment based on the predefined number of segmentations and the predefined length value of the search buffer includes: segmenting the total length of the original data according to the corresponding number of segmentations based on the interval in which the total length of the original data is located, to obtain the predefined length value of the original data segment.
[0011] In some embodiments, determining the length value of the original data segment based on the predefined number of segments and the predefined length value of the search buffer further includes: determining the length value of the original data segment based on the magnitude of the predefined length value of the original data segment and the predefined length value of the search buffer.
[0012] In some embodiments, determining the length value of the original data segment based on the size of the predefined length value of the original data segment and the predefined length value of the search buffer includes: in response to the predefined length value of the original data segment being less than the predefined length value of the search buffer, using the predefined length value of the search buffer as the length value of the original data segment.
[0013] In some embodiments, determining the length value of the original data segment based on the size of the predefined length value of the original data segment and the predefined length value of the search buffer further includes: in response to the predefined length value of the original data segment being not less than the predefined length value of the search buffer, using the predefined length value of the original data segment as the length value of the original data segment.
[0014] In some embodiments, updating the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment includes: in response to the existence of multiple longest repeating strings in the current segment, randomly selecting one of the multiple longest repeating strings to update the length value of the search buffer in the current segment.
[0015] In another aspect, the present invention provides an apparatus for improving the compression efficiency of a compression card, the apparatus comprising: a first module configured to determine the length value of the original data segment based on a predefined number of segmentations and a predefined length value of a search buffer; a second module configured to update the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment, thereby obtaining an updated length value of the search buffer corresponding to the current segment; a third module configured to compress the original data from the beginning position of the current segment according to the updated length value of the search buffer; and a fourth module configured to perform the next segmentation and compression from the end position where the current segment has been compressed, according to the segment length value, repeating until the original data has been compressed.
[0016] In some embodiments, the apparatus further includes a fourth module configured to update the length value of a pre-read buffer for storing uncompressed data based on the length value of the search buffer for the current segment, thereby obtaining an updated length value of the pre-read buffer corresponding to the current segment.
[0017] In some embodiments, the third module is further configured to: compress the original data according to the update length value of the search buffer corresponding to the current segment, the update length value of the pre-read buffer, and the starting position of the previous segment.
[0018] In some embodiments, the first module is further configured to: segment the total length of the original data according to the interval in which the total length of the original data is located, and obtain a predefined length value for the segmentation of the original data.
[0019] In some embodiments, the first module is further configured to: determine the length value of the original data segment based on the size of the predefined length value of the original data segment and the predefined length value of the search buffer.
[0020] In some embodiments, the first module is further configured to: in response to a predefined length value of the original data segment being less than a predefined length value of the search buffer, use the predefined length value of the search buffer as the length value of the original data segment.
[0021] In some embodiments, the first module is further configured to: in response to a predefined length value for the original data segment not being less than a predefined length value for the search buffer, use the predefined length value for the original data segment as the length value for the original data segment.
[0022] In some embodiments, the second module is further configured to: in response to the existence of multiple longest repeating strings in the current segment, randomly select one of the multiple longest repeating strings to update the length value of the search buffer in the current segment.
[0023] In another aspect of the present invention, a computer device is also provided, including at least one processor; and a memory storing computer instructions executable on the processor, the instructions, when executed by the processor, implementing the steps of any of the methods described above.
[0024] In another aspect of the present invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements any of the method steps described above.
[0025] The present invention has at least the following beneficial effects: The present invention provides a method, apparatus and device for improving the compression efficiency of a compression card. In the compression process, the original data to be compressed is divided into multiple data segments of unequal lengths for compression. Each segmentation requires recalculation of the search buffer length and the pre-read buffer length to ensure that both meet the conditions for the optimal compression ratio, thereby reducing the memory usage in the storage system and improving the data compression efficiency of the compression card. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.
[0027] Figure 1 A schematic diagram of an embodiment of a method for improving the compression efficiency of a compression card provided by the present invention is shown;
[0028] Figure 2 A schematic diagram of an embodiment of a device for improving the compression efficiency of a compression card provided by the present invention is shown;
[0029] Figure 3 A schematic diagram of an embodiment of a computer device provided by the present invention is shown;
[0030] Figure 4 A schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention is shown. Detailed Implementation
[0031] The following describes embodiments of the present invention. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various alternative forms.
[0032] Furthermore, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that comprises a list of elements may include not only those elements but also elements not expressly listed or inherent to such process, method, article, or apparatus.
[0033] One or more embodiments of this application will now be described with reference to the accompanying drawings.
[0034] Based on the above objectives, a first aspect of the present invention provides an embodiment of a method for improving the compression efficiency of a compression card. Figure 1 The diagram shown is an embodiment of a method for improving the compression efficiency of a compression card provided by the present invention. Figure 1 As shown, an embodiment of the present invention relates to a method for improving the compression efficiency of a compression card, which includes the following steps:
[0035] S1. Determine the length of the original data segment based on the predefined number of segments and the predefined length of the search buffer;
[0036] S2. Update the length value of the search buffer in the current segment according to the length value of the longest repeating string in the current segment, so as to obtain the updated length value of the search buffer corresponding to the current segment;
[0037] S3. Compress the original data from the starting position of the current segment according to the updated length value of the search buffer;
[0038] S4. Starting from the end position of the current segment compression, perform the next segmentation and compression according to the segment length value, repeating until the original data is compressed.
[0039] According to several embodiments of the present invention, the method further includes: updating the length value of the pre-read buffer used to store uncompressed data according to the length value of the search buffer of the current segment, so as to obtain the updated length value of the pre-read buffer corresponding to the current segment.
[0040] According to several embodiments of the present invention, compressing the original data from the starting position of the current segment according to the updated length value of the search buffer includes:
[0041] The original data is compressed based on the update length of the search buffer corresponding to the current segment, the update length of the prefetch buffer, and the starting position of the previous segment.
[0042] According to several embodiments of the present invention, determining the length value of the original data segment based on the predefined number of segmentations and the predefined length value of the search buffer includes: segmenting the total length of the original data according to the corresponding number of segmentations based on the interval in which the total length of the original data is located, thereby obtaining the predefined length value of the original data segment.
[0043] According to several embodiments of the present invention, determining the length value of the original data segment based on the predefined number of segments and the predefined length value of the search buffer further includes: determining the length value of the original data segment based on the magnitude of the predefined length value of the original data segment and the predefined length value of the search buffer.
[0044] According to several embodiments of the present invention, determining the length value of the original data segment based on the size of the predefined length value of the original data segment and the predefined length value of the search buffer includes: in response to the predefined length value of the original data segment being less than the predefined length value of the search buffer, using the predefined length value of the search buffer as the length value of the original data segment.
[0045] According to several embodiments of the present invention, determining the length value of the original data segment based on the size of the predefined length value of the original data segment and the predefined length value of the search buffer further includes: in response to the predefined length value of the original data segment being not less than the predefined length value of the search buffer, using the predefined length value of the original data segment as the length value of the original data segment.
[0046] According to several embodiments of the present invention, updating the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment includes: in response to the existence of multiple longest repeating strings in the current segment, randomly selecting one of the multiple longest repeating strings to update the length value of the search buffer in the current segment.
[0047] The following is another embodiment of a method for improving the compression efficiency of a compression card provided by the present invention.
[0048] Based on the total length L0 (in bytes) of the original data to be compressed, set the predefined length L of the search buffer. S L S The maximum value is 64K (unit: bytes), and a predefined denominator Y is defined. Y is segmented based on the interval containing the total length L0 of the original data, as shown in Table 1:
[0049] Predefined denominator Y <![CDATA[Total length L0 of the original data]]> 16 <= 1M bytes 800 <=50M bytes 8000 <= 500 MB 10000 >500M bytes
[0050] Table 1
[0051] The length of the search buffer is determined based on the initial length of the original data. If L0 / Y ≥ 64K, then the L of the search buffer... S =64K; if L0 / Y < 64K, then the L of the search buffer is... S =L0 / Y.
[0052] In this embodiment, the search buffer length is 64KB. For the original data to be compressed, a 64KB interval is extracted starting from position M. Within this interval, the longest repeating string is found and its length is recorded as L. The position N of the last character when the repeating string last appears is also recorded. Then, the search buffer length is updated to L = (N-M+1), and the look-ahead buffer length is updated to L based on the search buffer length. If two or more longest repeating strings are found, any one of them can be selected, as the longest repeating string has no impact on this application. Compression is performed based on the obtained search buffer length, look-ahead buffer length, and the starting position of the data to be compressed. Compression ends when the space in the search buffer is completely filled. Then, the next compression is performed according to the segmentation of the original data until all the source data to be compressed has been processed. When the search buffer is large enough, the optimal compression ratio is achieved when the size of the look-ahead buffer equals the "maximum repeating string length." However, even with a large enough look-ahead buffer, the optimal compression ratio is only achieved if the search buffer includes this maximum repeating string. Therefore, the look-ahead buffer length must be at least the search buffer length. Based on this, the original data to be compressed is segmented into multiple data segments of unequal length, and compression is performed segment by segment. Each segmentation requires recalculating the search buffer length and look-ahead buffer length to ensure they meet the optimal compression ratio condition. The entire compression process is segmented, and the search buffer length, look-ahead buffer length, and the length of the data being compressed for each segment are all variable. The search buffer length decreases relatively with each compression (compared to using a fixed value), reducing memory usage.
[0053] A second aspect of the present invention provides an apparatus for improving the compression efficiency of a compression card. Figure 2 The diagram shown is a schematic representation of an embodiment of a device for improving the compression efficiency of a compression card provided by the present invention. Figure 2As shown, the present invention provides an apparatus for improving the compression efficiency of a compression card, comprising: a first module 011 configured to determine the length value of the original data segment based on a predefined number of segmentations and a predefined length value of the search buffer; a second module 012 configured to update the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment, thereby obtaining an updated length value of the search buffer corresponding to the current segment; a third module 013 configured to compress the original data from the starting position of the current segment according to the updated length value of the search buffer; and a fourth module 014 configured to perform the next segmentation and compression from the ending position where the current segment has been compressed, according to the segment length value, repeating until the original data has been compressed.
[0054] According to several embodiments of the present invention, the apparatus further includes: a fourth module configured to update the length value of a pre-read buffer used to store uncompressed data based on the length value of the search buffer of the current segment, so as to obtain an updated length value of the pre-read buffer corresponding to the current segment.
[0055] According to several embodiments of the present invention, the third module is further configured to: compress the original data according to the update length value of the search buffer corresponding to the current segment, the update length value of the pre-read buffer, and the starting position of the previous segment.
[0056] According to several embodiments of the present invention, the first module is further configured to: segment the total length of the original data according to the interval in which the total length of the original data is located, and obtain a predefined length value for the segmentation of the original data.
[0057] According to several embodiments of the present invention, the first module is further configured to: determine the length value of the original data segment based on the size of the predefined length value of the original data segment and the predefined length value of the search buffer.
[0058] According to several embodiments of the present invention, the first module is further configured to: in response to a predefined length value of the original data segment being less than a predefined length value of the search buffer, use the predefined length value of the search buffer as the length value of the original data segment.
[0059] According to several embodiments of the present invention, the first module is further configured to: in response to a predefined length value of the original data segment not being less than a predefined length value of the search buffer, use the predefined length value of the original data segment as the length value of the original data segment.
[0060] According to several embodiments of the present invention, the second module is further configured to: in response to the existence of multiple longest repeating strings in the current segment, randomly select one of the multiple longest repeating strings to update the length value of the search buffer in the current segment.
[0061] To achieve the above objectives, a third aspect of the present invention provides a computer device. Figure 3 The diagram shown is a schematic representation of an embodiment of a computer device provided by the present invention. Figure 3 As shown, an embodiment of a computer device provided by the present invention includes the following modules: at least one processor 021; and a memory 022, the memory 022 storing computer instructions 023 executable on the processor 021, wherein the computer instructions 023, when executed by the processor 021, implement the steps of the method described above, wherein the steps include: determining the length value of the original data segment based on a predefined number of segmentations for the original data segmentation and a predefined length value of the search buffer; updating the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment, to obtain an updated length value of the search buffer corresponding to the current segment; compressing the original data from the beginning position of the current segment according to the updated length value of the search buffer; and performing the next segmentation and compression from the end position where the current segment has completed compression according to the segment length value, repeating until the original data has completed compression.
[0062] In some embodiments, the method further includes: updating the length value of a pre-read buffer used to store uncompressed data according to the length value of the search buffer of the current segment, so as to obtain the updated length value of the pre-read buffer corresponding to the current segment.
[0063] In some embodiments, compressing the original data from the starting position of the current segment according to the updated length value of the search buffer includes:
[0064] The original data is compressed based on the update length value of the search buffer corresponding to the current segment, the update length value of the pre-read buffer, and the starting position of the previous segment.
[0065] In some embodiments, determining the length value of the original data segment based on the predefined number of segmentations and the predefined length value of the search buffer includes: segmenting the total length of the original data according to the corresponding number of segmentations based on the interval in which the total length of the original data is located, to obtain the predefined length value of the original data segment.
[0066] In some embodiments, determining the length value of the original data segment based on the predefined number of segments and the predefined length value of the search buffer further includes: determining the length value of the original data segment based on the magnitude of the predefined length value of the original data segment and the predefined length value of the search buffer.
[0067] In some embodiments, determining the length value of the original data segment based on the size of the predefined length value of the original data segment and the predefined length value of the search buffer includes: in response to the predefined length value of the original data segment being less than the predefined length value of the search buffer, using the predefined length value of the search buffer as the length value of the original data segment.
[0068] In some embodiments, determining the length value of the original data segment based on the size of the predefined length value of the original data segment and the predefined length value of the search buffer further includes: in response to the predefined length value of the original data segment being not less than the predefined length value of the search buffer, using the predefined length value of the original data segment as the length value of the original data segment.
[0069] In some embodiments, updating the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment includes: in response to the existence of multiple longest repeating strings in the current segment, randomly selecting one of the multiple longest repeating strings to update the length value of the search buffer in the current segment.
[0070] The present invention also provides a computer-readable storage medium. Figure 4 The diagram shown is a schematic representation of an embodiment of a computer-readable storage medium provided by the present invention. Figure 4 As shown, computer-readable storage medium 031 stores a computer program 032 that is executed by a processor according to the above steps.
[0071] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for setting system parameters can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.
[0072] Furthermore, the method disclosed in the embodiments of the present invention can also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. When the computer program is executed by the processor, it performs the functions defined in the method disclosed in the embodiments of the present invention.
[0073] Furthermore, the above-described method steps and system units can also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to perform the functions of the above-described steps or units.
[0074] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.
[0075] In one or more exemplary designs, functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on or transmitted via a computer-readable medium. Computer-readable media include computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one location to another. Storage media may be any available medium accessible to a general-purpose or special-purpose computer. By way of example, and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other medium that may be used to carry or store the required program code in the form of instructions or data structures and is accessible to a general-purpose or special-purpose computer or a general-purpose or special-purpose processor. Furthermore, any connection may be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the aforementioned coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are all included in the definition of media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.
[0076] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.
[0077] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.
[0078] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0079] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0080] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.
Claims
1. A method for improving the compression efficiency of a compression card, characterized in that, include: Determining the length of a segment of the original data based on a predefined number of segmentations and a predefined length of a search buffer includes: segmenting the total length of the original data according to the corresponding number of segmentations based on the interval containing the total length of the original data, to obtain a predefined length of the segment; determining the length of the segment based on the magnitude of the predefined length of the segment and the predefined length of the search buffer; wherein, in response to the predefined length of the segment being less than the predefined length of the search buffer, the predefined length of the search buffer is used as the length of the segment; in response to the predefined length of the segment being not less than the predefined length of the search buffer, the predefined length of the segment is used as the length of the segment. The length of the search buffer in the current segment is updated based on the length of the longest repeating string in the current segment, so as to obtain the updated length value of the search buffer corresponding to the current segment; The original data is compressed from the starting position of the current segment according to the updated length value of the search buffer; Starting from the end position of the current segment compression, perform the next segmentation and compression according to the segment length value, repeating until the original data is fully compressed.
2. The method according to claim 1, characterized in that, The method further includes: The length of the pre-read buffer used to store uncompressed data is updated based on the length of the search buffer for the current segment, so as to obtain the updated length of the pre-read buffer corresponding to the current segment.
3. The method according to claim 2, characterized in that, The compression of the original data from the starting position of the current segment according to the updated length value of the search buffer includes: The original data is compressed based on the update length value of the search buffer corresponding to the current segment, the update length value of the pre-read buffer, and the starting position of the previous segment.
4. The method according to claim 1, characterized in that, The step of updating the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment includes: If there are multiple longest repeating strings in the current segment, one of the multiple longest repeating strings is randomly selected to update the length value of the search buffer in the current segment.
5. A device for improving the compression efficiency of a compression card, characterized in that, include: The first module is configured to determine the length of the original data segment based on a predefined number of segments and a predefined length value of the search buffer. The second module is configured to update the length value of the search buffer in the current segment based on the length value of the longest repeating string in the current segment, so as to obtain the updated length value of the search buffer corresponding to the current segment; The third module is configured to compress the original data from the starting position of the current segment according to the updated length value of the search buffer; The fourth module is configured to perform the next segmentation and compression from the termination position of the current segmentation according to the segment length value, repeating until the original data is compressed. The first module is further configured to segment the total length of the original data according to the interval in which the total length of the original data lies, according to the corresponding number of segmentations, to obtain a predefined length value for the segment of the original data; and to determine the length value of the segment of the original data according to the size of the predefined length value of the segment of the original data and the predefined length value of the search buffer; wherein, in response to the predefined length value of the segment of the original data being less than the predefined length value of the search buffer, the predefined length value of the search buffer is used as the length value of the segment of the original data; and in response to the predefined length value of the segment of the original data being not less than the predefined length value of the search buffer, the predefined length value of the segment of the original data is used as the length value of the segment of the original data.
6. A computer device, characterized in that, include: At least one processor; as well as A memory storing computer instructions executable on the processor, which, when executed by the processor, implement the steps of the method according to any one of claims 1-4.
Citation Information
Patent Citations
Heating power supply and demand information management platform based on big data
CN115204754A