Compression method, device, electronic device and storage medium
By setting the data quantity and compression ratio threshold during the data compression process, and compressing only when the conditions are met, the problem of increasing data quantity after compression is solved, and efficient utilization of resources is achieved.
Patent Information
- Application Number
- CN202111235097.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-22
- Publication Date
- 2025-09-02
- Estimated Expiration
- 2041-10-22
AI Technical Summary
During the data compression process, the prior art has the problem of increasing the amount of data after compression, resulting in wasting of storage and computing resources.
By calculating the comparison between the original data amount of the object to be compressed and the data amount threshold, the compression ratio is performed only when the original data amount is greater than the data amount threshold, and the compression ratio is calculated, and the target object is retained only when the compression ratio is not greater than the compression ratio threshold, avoiding unnecessary compression operations.
It effectively avoids the increase in compressed data volume due to too small data volume, saves computing resources, and makes rational use of storage resources, avoids waste of storage resources.
Smart Images

Figure CN113965206B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of data processing, and more particularly to the technical field of data compression, and can be applied to scenarios such as text, image, audio, and video compression. Background Art
[0002] Data compression is a technical method that reduces the amount of data to save storage space and improve transmission, storage, and processing efficiency without losing information. However, in some data compression scenarios, the compressed data volume actually increases, resulting in a waste of storage and computing resources. Summary of the Invention
[0003] The present disclosure provides a compression method, device, electronic device, and storage medium.
[0004] According to a first aspect of the present disclosure, a compression method is provided, comprising:
[0005] In response to a compression triggering event for the object to be compressed, calculating the original data volume of the object to be compressed;
[0006] Comparing the original data volume of the object to be compressed with a data volume threshold, and compressing the object to be compressed into a target object when it is determined that the original data volume of the object to be compressed is greater than the data volume threshold;
[0007] Calculate the compression ratio of the target object;
[0008] When it is determined that the compression ratio of the target object is not greater than the compression ratio threshold, the target object is retained.
[0009] According to a second aspect of the present disclosure, there is provided a compression device comprising:
[0010] A data volume calculation module, configured to calculate the original data volume of the object to be compressed in response to a compression triggering event for the object to be compressed;
[0011] A data compression module is configured to compare the original data volume of the object to be compressed with a data volume threshold, and compress the object to be compressed into a target object when it is determined that the original data volume of the object to be compressed is greater than the data volume threshold;
[0012] A compression ratio calculation module, used to calculate the compression ratio of the target object;
[0013] The compression result evaluation module is used to retain the target object when it is determined that the compression ratio of the target object is not greater than the compression ratio threshold.
[0014] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0015] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned compression method.
[0016] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned compression method.
[0017] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the above-mentioned compression method when executed by a processor.
[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description.
[0019] The beneficial effects of the technical solution provided by this disclosure are:
[0020] In the technical solution disclosed in the present invention, compression operation is performed on the compressed object only when it is determined that the original data volume of the object to be compressed is greater than the data volume threshold. This can largely avoid the increase in the compressed data volume due to the small data volume of the object to be compressed, and avoid performing compression operations too many times with compression benefits less than expected, which can effectively save computing resources. Moreover, after obtaining the compressed target object, the target object is retained only when the compression ratio of the target object is not greater than the compression ratio threshold, thereby ensuring that the retained target object meets the expected compression benefit, thereby reasonably and efficiently utilizing storage resources and largely avoiding the waste of storage resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.
[0022] Figure 1 A schematic diagram of a compression method according to an embodiment of the present disclosure is shown;
[0023] Figure 2 A schematic diagram showing a flow chart of another compression method provided by an embodiment of the present disclosure is shown;
[0024] Figure 3 One of the schematic diagrams of a compression device provided by an embodiment of the present disclosure is shown;
[0025] Figure 4 A second schematic diagram of a compression device provided by an embodiment of the present disclosure is shown;
[0026] Figure 5 A schematic block diagram of an example electronic device that can be used to implement the compression method provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0027] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0028] Data compression refers to a technical method that reduces the amount of data to reduce storage space and improve its transmission, storage, and processing efficiency without losing information. However, in some scenarios where data is compressed, the amount of compressed data will actually increase, which will result in a waste of storage and computing resources. Specifically, compressing data will generate some additional dictionary data. For original data with a small amount of data, after compressing the original data, the compressed data may be larger than the original data due to the existence of additional dictionary data, and the compression benefit will be less than expected. Moreover, data compression itself is a relatively high-cost algorithm. Performing a compression operation that does not provide the expected compression benefit is also a waste of computing resources.
[0029] The compression method, device, electronic device, and storage medium provided by the embodiments of the present disclosure are intended to solve at least one of the above technical problems in the prior art.
[0030] Figure 1 A schematic diagram of a compression method provided by an embodiment of the present disclosure is shown in FIG. Figure 1 As shown, the method may mainly include the following steps:
[0031] S110: In response to a compression triggering event for the object to be compressed, calculating the original data volume of the object to be compressed.
[0032] In the embodiments of the present disclosure, the data desired to be compressed in the current scenario is referred to as a to-be-compressed object. The to-be-compressed object can be any type of data, such as text data, image data, audio data, and video data. The compression triggering event can be determined based on the actual situation. For example, the compression triggering event can be a user-triggered operation to compress the to-be-compressed object, or it can be a related event executed by the system based on a preset judgment condition (e.g., a judgment condition related to time).
[0033] S120: Compare the original data volume of the object to be compressed with a data volume threshold, and when it is determined that the original data volume of the object to be compressed is greater than the data volume threshold, compress the object to be compressed into a target object.
[0034] For ease of understanding and expression, the embodiment of the present disclosure defines the data volume of the object to be compressed as the original data volume, and defines the data obtained after compressing the object to be compressed as the target object.
[0035] The data volume threshold disclosed in the present invention can be determined based on actual design needs. It can be understood that when the original data volume of the object to be compressed is greater than the data volume threshold, it indicates that the object to be compressed is data with a large data volume. After compressing the object to be compressed, the data volume of the target object obtained is likely to be smaller than the original data volume of the object to be compressed.
[0036] Optionally, the disclosed embodiment may terminate the compression process for the object to be compressed when it is determined that the original data size of the object to be compressed is not greater than a data size threshold. It is understood that when the original data size of the object to be compressed is less than the data size threshold, it indicates that the object to be compressed is of relatively small size. After compressing the object to be compressed, the resulting target object will have a smaller data size than the original data size of the object to be compressed, resulting in a lower compression benefit than expected. Therefore, there is no need to compress the object to be compressed to save computing resources.
[0037] The embodiment of the present disclosure can use a corresponding compression algorithm to compress the object to be compressed into a target object according to actual needs. The embodiment of the present disclosure does not limit the type of compression algorithm, and the compression algorithm can be a zstd algorithm, a zlib algorithm, a gzip algorithm, etc.
[0038] S130: Calculate the compression ratio of the target object.
[0039] It should be noted that the compression ratio of the target object is the ratio of the data size of the target object to the original data size of the object to be compressed. In this step, the data size of the target object can be calculated and then divided by the original data size of the object to be compressed to obtain the compression ratio.
[0040] When the target object data size is larger than the original data size of the object to be compressed, the compression ratio is greater than 1; when the target object data size is smaller than the original data size of the object to be compressed, the compression ratio is between 0 and 1; and when the target object data size is equal to the original data size of the object to be compressed, the compression ratio is greater than 1. It can be understood that when the compression ratio is between 0 and 1, it indicates a positive compression gain.
[0041] S140: When it is determined that the compression ratio of the target object is not greater than the compression ratio threshold, retain the target object.
[0042] The compression ratio threshold may be determined based on actual design requirements. It is understood that in order to ensure that the compression gain of the retained target object is positive, the compression ratio threshold is usually not greater than 1. When the compression ratio of the target object is not greater than the compression ratio threshold, it indicates that the compression gain of the target object has met expectations.
[0043] Optionally, when the embodiment of the present disclosure determines that the compression ratio of the target object is greater than the compression ratio threshold, the target object may be deleted.
[0044] It can be understood that when the compression ratio of a target object exceeds the compression ratio threshold, the compression benefit of the target object is determined to be less than expected, and the target object is deleted to avoid excessive storage resource usage. For example, if the compression ratio threshold is set to 1, when the compression ratio threshold is generally not greater than 1, it means that the data volume of the target object is actually greater than the original data volume of the data to be compressed, and the compression benefit of the target object is negative. Therefore, the target object should be deleted, and the uncompressed file to be compressed should be retained, thereby avoiding wasting storage resources on the target object with a negative compression benefit.
[0045] The compression method provided by the embodiment of the present disclosure performs compression operation on the compressed object only when it is determined that the original data volume of the object to be compressed is greater than the data volume threshold. This can avoid, to a large extent, the increase in the compressed data volume due to the small data volume of the object to be compressed, and avoids performing compression operations too many times with compression benefits less than expected, which can effectively save computing resources. Moreover, after obtaining the compressed target object, the target object is retained only when the compression ratio of the target object is not greater than the compression ratio threshold, thereby ensuring that the retained target object meets the expected compression benefit, thereby reasonably and efficiently utilizing storage resources and avoiding, to a large extent, the waste of storage resources.
[0046] The disclosed embodiment also updates and adjusts the data volume threshold, for example, the data volume threshold can be increased or decreased, so as to more accurately determine the compression objects whose compression benefits meet expectations based on the data volume threshold, thereby further reducing the number of compression operations whose compression benefits are lower than expected and saving computing resources.
[0047] Optionally, the embodiment of the present disclosure may increase the data volume threshold after deleting the target object.
[0048] Optionally, the embodiment of the present disclosure can, after comparing the original data volume of the object to be compressed with a data volume threshold, randomly compress the object to be compressed into a target object with a preset probability when it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold; calculate the compression ratio of the target object; and reduce the data volume threshold when it is determined that the compression ratio of the target object is greater than the compression ratio threshold.
[0049] Figure 2A schematic diagram of a compression method provided by an embodiment of the present disclosure is shown in FIG. Figure 2 As shown, the method may mainly include the following steps:
[0050] S210: In response to a compression triggering event for the object to be compressed, calculating the original data volume of the object to be compressed.
[0051] In the embodiments of the present disclosure, the data desired to be compressed in the current scenario is referred to as a to-be-compressed object. The to-be-compressed object can be any type of data, such as text data, image data, audio data, and video data. The compression triggering event can be determined based on the actual situation. For example, the compression triggering event can be a user-triggered operation to compress the to-be-compressed object, or it can be a related event executed by the system based on a preset judgment condition (e.g., a judgment condition related to time).
[0052] S220: Compare the original data volume of the object to be compressed with the data volume threshold.
[0053] For ease of understanding and description, the disclosed embodiments define the data volume of the object to be compressed as the original data volume, and the data obtained after compressing the object to be compressed as the target object. The data volume threshold of the disclosed embodiments can be determined based on actual design requirements. This step can compare the original data volume of the object to be compressed with the data volume threshold.
[0054] S230: When it is determined that the original data volume of the object to be compressed is greater than the data volume threshold, compress the object to be compressed into a target object.
[0055] It can be understood that when the original data volume of the object to be compressed is greater than the data volume threshold, it indicates that the object to be compressed is data with a large data volume. After compressing the object to be compressed, the data volume of the obtained target object is likely to be smaller than the original data volume of the object to be compressed.
[0056] S240: When it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold, the compression process for the object to be compressed is terminated.
[0057] Optionally, the disclosed embodiment may terminate the compression process for the object to be compressed when it is determined that the original data size of the object to be compressed is not greater than a data size threshold. It is understood that when the original data size of the object to be compressed is less than the data size threshold, it indicates that the object to be compressed is of relatively small size. After compressing the object to be compressed, the resulting target object will have a smaller data size than the original data size of the object to be compressed, resulting in a lower compression benefit than expected. Therefore, there is no need to compress the object to be compressed to save computing resources.
[0058] S250: Calculate the compression ratio of the target object.
[0059] It should be noted that the compression ratio of the target object is the ratio of the target object's data size to the original data size of the object to be compressed. In this step, the target object's data size can be calculated and then divided by the original data size of the object to be compressed to obtain the compression ratio. After step S25, step S260 or step S270 can be executed.
[0060] When the target object data size is larger than the original data size of the object to be compressed, the compression ratio is greater than 1; when the target object data size is smaller than the original data size of the object to be compressed, the compression ratio is between 0 and 1; and when the target object data size is equal to the original data size of the object to be compressed, the compression ratio is greater than 1. It can be understood that when the compression ratio is between 0 and 1, it indicates a positive compression gain.
[0061] S260: When it is determined that the compression ratio of the target object is not greater than the compression ratio threshold, retain the target object.
[0062] In the disclosed embodiments, the compression ratio threshold can be determined based on actual design requirements. It is understood that to ensure a positive compression gain for the retained target object, the compression ratio threshold is typically no greater than 1. When the compression ratio of the target object is no greater than the compression ratio threshold, it indicates that the compression gain for the target object has met expectations, and the target object can be retained, and the compression process for the target object to be compressed ends.
[0063] S270: When it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the target object is deleted.
[0064] It can be understood that when the compression ratio of a target object exceeds the compression ratio threshold, the compression benefit of the target object is determined to be less than expected, and the target object is deleted to avoid excessive storage resource usage. For example, if the compression ratio threshold is set to 1, when the compression ratio threshold is generally not greater than 1, it means that the data volume of the target object is actually greater than the original data volume of the data to be compressed, and the compression benefit of the target object is negative. Therefore, the target object should be deleted, and the uncompressed file to be compressed should be retained, thereby avoiding wasting storage resources on the target object with a negative compression benefit.
[0065] It can be understood that when the compression ratio of the target object is greater than the compression ratio threshold, it indicates that the data volume threshold needs to be further increased so that the compression ratio threshold can be more accurately determined based on the data volume threshold to ensure that the compression gain can meet the expected object to be compressed, thereby further reducing the number of compression operations where the compression gain is lower than expected and saving computing resources.
[0066] S280: Increase the data volume threshold.
[0067] It can be understood that after increasing the data volume threshold, more data to be compressed can be made smaller than the data volume threshold, thereby reducing the number of compression operations where the compression benefit is lower than expected and saving computing resources.
[0068] Optionally, in the scenario of increasing the data volume threshold, the embodiment of the present disclosure can set the compression ratio threshold to 1. Since the compression ratio of the target object is greater than the compression ratio threshold, the data volume of the target object is greater than the original data volume of the object to be compressed. Therefore, the original data volume of the object to be compressed can be used as the new data volume threshold, thereby increasing the data volume threshold.
[0069] Optionally, in the scenario of increasing the data volume threshold, the embodiment of the present disclosure adjusts the data volume threshold according to a preset increase ratio, and uses the adjusted data volume threshold as the new data volume threshold. Here, the increase ratio can be determined according to actual design requirements. For example, the increase ratio can be 1%, and the original data volume threshold is increased by 1% to serve as the new data volume threshold.
[0070] To avoid the data volume threshold being updated to an unreasonably large value due to multiple increases in the data volume threshold, a probabilistic reduction method can be used to make it possible to reduce the data volume threshold. Optionally, after comparing the original data volume of the object to be compressed with the data volume threshold, the disclosed embodiment can randomly compress the object to be compressed into a target object with a preset probability when it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold, and then calculate the compression ratio of the target object; when it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the data volume threshold is reduced.
[0071] The value of the aforementioned preset probability can be determined based on actual design requirements. For example, the preset probability can be 1%, 1 / 1000, or 1 / 10,000. With a preset probability of 1 / 1000, one object to be compressed can be selected from one thousand objects to be compressed whose original data volume is not greater than the data volume threshold, and the selected object to be compressed can be used as the target object. The compression ratio of the target object is calculated, and if it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the data volume threshold is reduced.
[0072] Optionally, in the scenario of reducing the data volume threshold, the embodiment of the present disclosure can set the compression ratio threshold to 1. Since the compression ratio of the target object is less than the compression ratio threshold, the data volume of the target object is less than the original data volume of the object to be compressed. Therefore, the original data volume of the object to be compressed can be used as the new data volume threshold, thereby reducing the data volume threshold.
[0073] Optionally, in the scenario of reducing the data volume threshold, the embodiment of the present disclosure adjusts the data volume threshold according to a preset reduction ratio, and uses the adjusted data volume threshold as the new data volume threshold. Here, the reduction ratio can be determined according to actual design requirements. For example, the reduction ratio can be 1%, and the original data volume threshold is reduced by 1% to serve as the new data volume threshold.
[0074] It should be noted here that the embodiment of the present disclosure can use the corresponding compression algorithm to compress the object to be compressed into the target object according to actual needs. The embodiment of the present disclosure does not limit the type of compression algorithm. The compression algorithm can be the zstd algorithm, zlib algorithm, gzip algorithm, etc.
[0075] Based on the same principle as the compression method above, Figure 3 shows one of the schematic diagrams of a compression device provided by an embodiment of the present disclosure, Figure 4 FIG2 shows a second schematic diagram of a compression device provided by an embodiment of the present disclosure. Figure 3 As shown, the compression device 300 includes a data volume calculation module 310, a data compression module 320, a compression ratio calculation module 330 and a compression result evaluation module 340.
[0076] The data volume calculation module 310 is configured to calculate the original data volume of the object to be compressed in response to a compression triggering event for the object to be compressed.
[0077] The data compression module 320 is configured to compare the original data volume of the object to be compressed with a data volume threshold, and compress the object to be compressed into a target object when it is determined that the original data volume of the object to be compressed is greater than the data volume threshold.
[0078] The compression ratio calculation module 330 is used to calculate the compression ratio of the target object.
[0079] The compression result evaluation module 340 is configured to retain the target object when it is determined that the compression ratio of the target object is not greater than the compression ratio threshold.
[0080] The compression device provided by the embodiment of the present disclosure performs compression operation on the compressed object only when it determines that the original data volume of the object to be compressed is greater than the data volume threshold. This can largely avoid the increase in the compressed data volume due to the small data volume of the object to be compressed, avoid performing compression operations too many times with compression benefits less than expected, and can effectively save computing resources. Moreover, after obtaining the compressed target object, the target object is retained only when the compression ratio of the target object is not greater than the compression ratio threshold, thereby ensuring that the retained target object meets the expected compression benefit, thereby reasonably and efficiently utilizing storage resources and largely avoiding the waste of storage resources.
[0081] In the embodiment of the present disclosure, the compression result evaluation module 340 is further configured to delete the target object when it is determined that the compression ratio of the target object is greater than a compression ratio threshold.
[0082] In the embodiment of the present disclosure, Figure 4 As shown, the compression device further includes a threshold adjustment module 350, and the threshold adjustment module 350 is used to increase the data volume threshold.
[0083] In the embodiment of the present disclosure, when the threshold adjustment module 350 is used to increase the data volume threshold, it is specifically used to: use the original data volume of the object to be compressed as the new data volume threshold; or adjust the data volume threshold according to a preset increase ratio, and use the adjusted data volume threshold as the new data volume threshold.
[0084] In the embodiment of the present disclosure, the data compression module 320 is further configured to terminate the compression process for the object to be compressed when it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold.
[0085] In the embodiment of the present disclosure, the threshold adjustment module 350 is also used to: when it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold, randomly compress the object to be compressed into a target object with a preset probability; calculate the compression ratio of the target object; when it is determined that the compression ratio of the target object is greater than the compression ratio threshold, reduce the data volume threshold.
[0086] In the embodiment of the present disclosure, when the threshold adjustment module 350 is used to reduce the data volume threshold, it is specifically used to: use the original data volume of the object to be compressed as the new data volume threshold; or adjust the data volume threshold according to a preset reduction ratio, and use the adjusted data volume threshold as the new data volume threshold.
[0087] It is understood that the above-mentioned modules of the compression device in the embodiment of the present disclosure have the function of implementing the corresponding steps of the above-mentioned compression method. This function can be implemented by hardware, or it can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions. The above-mentioned modules can be software and / or hardware, and each of the above-mentioned modules can be implemented individually or integrated into multiple modules. For a detailed description of the functions of the modules of the above-mentioned compression device, please refer to the corresponding description of the above-mentioned compression method, and will not be repeated here.
[0088] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0089] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0090] Figure 5A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0091] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0092] Various components in device 500 are connected to I / O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, optical disk, etc.; and a communication unit 509, such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.
[0093] The computing unit 501 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the compression method. For example, in some embodiments, the compression method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the compression method described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the compression method by any other appropriate means (e.g., by means of firmware).
[0094] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0095] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0096] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0098] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.
[0099] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.
[0100] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.
[0101] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.
Claims
1. A compression method, comprising: In response to a compression triggering event for an object to be compressed, calculating the original data volume of the object to be compressed; comparing the original data volume of the object to be compressed with a data volume threshold, and compressing the object to be compressed into a target object when it is determined that the original data volume of the object to be compressed is greater than the data volume threshold; Calculating a compression ratio of the target object; When it is determined that the compression ratio of the target object is not greater than the compression ratio threshold, retaining the target object; After comparing the original data volume of the object to be compressed with the data volume threshold, the method further includes: When it is determined that the original data amount of the object to be compressed is not greater than the data amount threshold, randomly compressing the object to be compressed into a target object with a preset probability; Calculating a compression ratio of the target object; When it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the data volume threshold is reduced.
2. The method according to claim 1, after calculating the compression ratio of the target object, further comprising: When it is determined that the compression ratio of the target object is greater than the compression ratio threshold, the target object is deleted.
3. The method according to claim 2, further comprising, after deleting the target object: Increase the data volume threshold.
4. The method according to claim 3, wherein: The increasing the data volume threshold comprises: Taking the original data volume of the object to be compressed as a new data volume threshold; Alternatively, the data volume threshold is adjusted according to a preset increase ratio, and the adjusted data volume threshold is used as the new data volume threshold.
5. The method according to claim 1, after comparing the original data volume of the object to be compressed with the data volume threshold, further comprising: When it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold, the compression process for the object to be compressed is ended.
6. The method according to claim 1, wherein The reducing the data amount threshold comprises: Taking the original data volume of the object to be compressed as a new data volume threshold; Alternatively, the data volume threshold is adjusted according to a preset reduction ratio, and the adjusted data volume threshold is used as the new data volume threshold.
7. A compression device comprising: a data volume calculation module, configured to calculate the original data volume of the object to be compressed in response to a compression triggering event for the object to be compressed; a data compression module, configured to compare the original data volume of the object to be compressed with a data volume threshold, and compress the object to be compressed into a target object when it is determined that the original data volume of the object to be compressed is greater than the data volume threshold; A compression ratio calculation module, used to calculate the compression ratio of the target object; a compression result evaluation module, configured to retain the target object when determining that the compression ratio of the target object is not greater than a compression ratio threshold; a threshold adjustment module, configured to randomly compress the object to be compressed into a target object with a preset probability when it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold; Calculating the compression ratio of the target object; and reducing the data volume threshold when determining that the compression ratio of the target object is greater than the compression ratio threshold.
8. The device according to claim 7, wherein the compression result evaluation module is further configured to: delete the target object when it is determined that the compression ratio of the target object is greater than the compression ratio threshold. 9 . The apparatus according to claim 8 , further comprising a threshold adjustment module, configured to increase the data volume threshold.
10. The device according to claim 9, wherein When the threshold adjustment module is used to increase the data volume threshold, it is specifically used to: Taking the original data volume of the object to be compressed as a new data volume threshold; Alternatively, the data volume threshold is adjusted according to a preset increase ratio, and the adjusted data volume threshold is used as the new data volume threshold.
11. The device according to claim 7, wherein the data compression module is further configured to: terminate the compression process for the object to be compressed when it is determined that the original data volume of the object to be compressed is not greater than the data volume threshold.
12. The device according to claim 7, wherein When the threshold adjustment module is used to reduce the data volume threshold, it is specifically used to: Taking the original data volume of the object to be compressed as a new data volume threshold; Alternatively, the data volume threshold is adjusted according to a preset reduction ratio, and the adjusted data volume threshold is used as the new data volume threshold.
13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.
15. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Read-write method, system and device of network file system and medium
CN112632021A
Avoiding compression of high-entropy data during creation of a backup of a source storage
US9626249B1