Data compression method and device, computer equipment and storage medium
By monitoring the compression rate of memory pages in real time and adjusting the compression strategy dynamically, the problem of memory pages in the existing technology is solved, the compression efficiency and effect are improved, and the waste of computing resources is avoided.
Patent Information
- Application Number
- CN202310541834.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
When the prior art compresses difficult to compress memory pages, it leads to wasting of computing resources and cannot evaluate and adjust the compression progress in real time, limiting the improvement of compression efficiency.
When compressing the target memory page, the compression rate is obtained and monitored in real time. If the compression rate is higher than or equal to the preset threshold, the compression will be stopped; if the compression rate is lower than the threshold, the compression will continue and dynamically adjust the compression strategy to improve efficiency and Effect.
It effectively avoids invalid compression, saves computing resources and time, improves the efficiency of memory space release, and improves the compression effect while ensuring that resources are not wasted.
Smart Images

Figure CN118944677A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of memory compression technology, and in particular to a data compression method, device, computer equipment and storage medium. Background Art
[0002] The device will allocate certain memory pages according to the needs of the application. Even when the application is not running, it will occupy a part of the memory pages. When the device memory is tight, the memory compression mechanism will be used to compress the memory pages occupied by the application and write the compressed memory pages to the compression space first, which can release part of the memory space. When the data is needed again, it will be decompressed and called.
[0003] However, there are some data that are difficult to achieve significant compression effects even after compression processing. Relevant technologies will still try to fully compress these data, but in fact, invalid compression is performed, which leads to a waste of system resources of the device during the compression process. Summary of the invention
[0004] The present application proposes a data compression method, apparatus, computer equipment and storage medium to reduce the probability of invalid compression.
[0005] In a first aspect, an embodiment of the present application provides a data compression method, the method comprising: in the process of compressing a target memory page, obtaining the real-time compression rate of the first data that has completed the compression processing in the target memory page; if the real-time compression rate is greater than or equal to the first compression rate, stopping the compression processing of the target memory page; if the real-time compression rate is less than the first compression rate, continuing the compression processing of the target memory page.
[0006] In a second aspect, an embodiment of the present application provides a data compression device, the device comprising: a compression rate acquisition module and a compression processing module. The compression rate acquisition module is used to obtain the real-time compression rate of the first data that has completed the compression processing in the target memory page during the process of compressing the target memory page; the compression processing module is used to stop compressing the target memory page if the real-time compression rate is greater than or equal to the first compression rate; if the real-time compression rate is less than the first compression rate, continue to compress the target memory page.
[0007] In a third aspect, an embodiment of the present application provides a computer device, comprising: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.
[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the above method.
[0009] In the scheme provided by the present application, in the process of compressing the target memory page, the real-time compression rate of the first data that has completed the compression processing in the target memory page is obtained; if the real-time compression rate is greater than or equal to the first compression rate, the compression processing of the target memory page is stopped; if the real-time compression rate is less than the first compression rate, the compression processing of the target memory page continues. In this way, in the process of compressing the memory page, the real-time compression rate can be monitored, and when the real-time compression rate is high, the compression process is terminated in advance, avoiding the occurrence of problems such as the consumption of computing resources of the device due to the continued compression when it is difficult to obtain an effective compression effect; when the real-time compression rate is not high, the target memory page is tried to continue to be compressed to achieve a better compression effect and release more memory space. It can be seen that the real-time compression effect can be obtained according to the real-time compression rate, and then the compression strategy can be dynamically adjusted, while ensuring that computing resources are not wasted, the compression efficiency and compression effect are improved as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0011] Figure 1 A flow chart of a data compression method provided in an embodiment of the present application is shown.
[0012] Figure 2 A flow chart of a data compression method provided in another embodiment of the present application is shown.
[0013] Figure 3 Shows Figure 2 A flowchart of the sub-steps of step S230 in one implementation manner.
[0014] Figure 4 A flow chart of a data compression method provided in yet another embodiment of the present application is shown.
[0015] Figure 5 It is a block diagram of a data compression device provided according to an embodiment of the present application.
[0016] Figure 6 It is a block diagram of a computer device according to an embodiment of the present application for executing the data compression method according to an embodiment of the present application.
[0017] Figure 7 It is a storage unit of an embodiment of the present application for storing or carrying a program code for implementing a data compression method according to an embodiment of the present application. DETAILED DESCRIPTION
[0018] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0019] It should be noted that in some processes described in the specification, claims and the above-mentioned drawings of the present application, multiple operations that appear in a specific order are included, and these operations may not be executed or executed in parallel in the order in which they appear in this article. The sequence numbers of operations such as S110, S120, etc. are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or less operations, and these operations may be executed in sequence or in parallel. And, the terms "first", "second", etc. in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.
[0020] In related technologies, the memory compression mechanism is a mechanism that compresses memory pages and writes them into the memory compression space. When the memory compression triggering conditions are met, for example, when the system memory is tight, the memory page with the least number of uses is selected from the target linked list for memory compression, and then the compressed memory page is written into the memory compression space, thereby releasing part of the memory space to meet the system memory requirements.
[0021] However, due to some characteristics of the data in the memory page, such as high randomness and no repeated substrings, it is difficult to achieve significant compression effects even after the data with such characteristics has been completely compressed. However, the memory compression mechanism of the related art still attempts to completely compress these data, resulting in a waste of computing resources. Moreover, the compression algorithm in the related art can only determine the final compression rate after the compression process is completed on the entire data block, which means that the compression progress cannot be evaluated and adjusted in real time, which also limits the improvement of compression efficiency.
[0022] The inventor proposes a data compression method, device, computer equipment and storage medium. In the process of compressing the target memory page, the compression process is adjusted in real time in combination with the real-time compression rate to reduce the probability of invalid compression and improve the compression efficiency. The data compression method provided by the embodiment of the present application is described in detail below.
[0023] Please refer to Figure 1 , Figure 1 A flowchart of a data compression method provided in an embodiment of the present application is shown below. Figure 1 The data compression method provided in the embodiment of the present application is described in detail. The data compression method may include the following steps:
[0024] Step S110: in the process of compressing the target memory page, obtaining the real-time compression rate of the first data that has completed the compression process in the target memory page.
[0025] In this embodiment, the first data is part of the data that has been compressed during the process of compressing the data in the target memory page. Specifically, the ratio of the amount of the first data in the target memory page before compression to the amount of the first data in the target memory page after compression can be obtained as the real-time compression rate of the first data.
[0026] For example, the data size of the first data in the target memory page before compression is 100 kilobytes (kB), and the data size of the first data in the target memory page after compression is 95kB, then it can be determined that the real-time compression rate for the first data is 95%.
[0027] In some implementations, since it is necessary to detect the compression effect of the target memory page in real time during the compression process of the target memory page, the real-time compression rate of the first data that has completed the compression process in the target memory page can be obtained each time the fourth target data volume is compressed during the compression process of the target memory page. The fourth target data volume is a fixed data volume for each compression process in the memory compression mechanism, for example, 4kB; that is, during the compression process of the target memory page, after each 4kB of data is compressed, the real-time compression rate of the first data that has completed the compression process in the target memory page is obtained to achieve the purpose of real-time detection of the compression rate.
[0028] The algorithm for compressing the target memory page is a lossless data compression algorithm, which can be a dictionary-type compression algorithm. The dictionary-type compression algorithm replaces a string that has appeared in the dictionary with an index value to achieve the purpose of compression. In addition, since the dictionary-type compression algorithm scans the data in the target memory page one by one in sequence, the compression process can be completed, thereby reducing the compression time cost.
[0029] Step S120: If the real-time compression ratio is greater than or equal to the first compression ratio, stop compressing the target memory page.
[0030] In this embodiment, the first compression rate may be a preset value, for example, 90%; of course, the value may be adjusted accordingly according to actual needs, and this embodiment does not limit this.
[0031] It can be understood that the compression effect is negatively correlated with the compression rate, that is, the greater the compression rate, the worse the compression effect, and the smaller the compression rate, the better the compression effect. Therefore, after obtaining the real-time compression rate for the first data, the implemented compression rate can be compared with the first compression rate. If the real-time compression rate is greater than or equal to the first compression rate, it means that the compression rate of the first data that has completed the compression processing is large and the compression effect is poor. Since the data characteristics stored in the same memory page are generally similar, if the data that has not completed the compression processing in the target memory page is still compressed at this time, the compression effect will be similar to that of the first data, which is relatively poor. Therefore, when it is determined that the real-time compression rate is greater than or equal to the first compression rate, the compression processing of the target memory page can be stopped. For example, if the data in the target memory page is highly random or has no repeated substrings and other characteristics, after the compression processing of part of the data in the target memory page is completed, it is detected that the compression rate for the part of the data is large and the compression effect is poor, then the continued compression of the target memory page can be stopped. It can be seen that the compression process can be terminated in advance according to the real-time compression rate during the compression process, which helps to avoid problems such as wasting the computing resources of the device due to continuing compression when it is difficult to obtain an effective compression effect; saving time costs and computing resources of the device.
[0032] Step S130: If the real-time compression rate is less than the first compression rate, continue to compress the target memory page.
[0033] Optionally, if the real-time compression rate is less than the first compression rate, it indicates that the compression effect on the first data is not very bad. Therefore, the uncompressed data in the target memory page can continue to be compressed to achieve a better compression effect and release more memory space.
[0034] The data compression method provided in this application can be applied to memory storage systems such as file systems, databases, or cloud storage. After compressing the data in the memory page, it can effectively save storage space, improve storage utilization, and reduce the cost of data transmission and access. Of course, it can also be applied to data backup and archiving systems. By compressing infrequently accessed data, it can save backup and archiving storage space and reduce storage costs. This embodiment does not limit this.
[0035] In this embodiment, during the compression process of the memory page, the real-time compression rate can be monitored. When the real-time compression rate is high, the compression process is terminated in advance, thereby avoiding the occurrence of problems such as continued compression leading to consumption of computing resources of the device when it is difficult to obtain an effective compression effect; when the real-time compression rate is not high, the target memory page is attempted to be compressed to achieve a better compression effect and release more memory space. It can be seen that the real-time compression effect can be obtained according to the real-time compression rate, and the compression strategy can be dynamically adjusted, thereby ensuring that computing resources are not wasted and the compression efficiency and compression effect are improved as much as possible.
[0036] Please refer to Figure 2 , Figure 2 A flowchart of a data compression method provided by another embodiment of the present application is shown below. Figure 2 The data compression method provided in the embodiment of the present application is described in detail. The data compression method may include the following steps:
[0037] Step S210: in the process of compressing the target memory page, obtaining the real-time compression rate of the first data that has completed the compression process in the target memory page.
[0038] In this embodiment, the specific implementation of step S210 can refer to the content of the above-mentioned embodiment, which will not be repeated here.
[0039] Step S220: If the real-time compression rate is less than the first compression rate, the current average compression rate for the target memory page is obtained.
[0040] It is understandable that if the real-time compression rate is less than the first compression rate, it indicates that the real-time compression effect is not very bad, so the target memory page can continue to be compressed. However, before continuing to compress the target memory page, the current average compression rate for the target memory page can be obtained, and then whether the target memory page needs to be further compressed can be more comprehensively determined based on the average compression rate.
[0041] Specifically, the ratio between the amount of all data in the target memory page after the compression process for the first data is completed and the amount of all data in the target memory page before compression is obtained as the average compression rate. For example, if the amount of all data in the target memory page after the compression process for the first data is 40kB, and the amount of all data in the target memory page before compression is 100kB, it can be determined that the current average compression rate for the target memory page is 40%.
[0042] Step S230: If the average compression ratio is greater than the second compression ratio, continue to compress the second data in the target memory page, the second compression ratio is less than the first compression ratio, and the second data is other data in the target memory page except the first data.
[0043] In this embodiment, the second compression rate can be a preset value smaller than the first compression rate, for example, 50%. The first compression rate can be generally understood as the maximum compression rate threshold for determining the compression effect during the compression process, and the second compression rate can be generally understood as the compression rate expected to be achieved after the compression is completed. The second data is other data in the target memory page except the first data that has been compressed, that is, data that has not yet been compressed.
[0044] In some embodiments, see Figure 3 , step S230 may include the following contents from step S231 to step S235:
[0045] Step S231: If the average compression rate is greater than the second compression rate, obtain the product of the data volume of the second data in the target memory page and the real-time compression rate to obtain a first target data volume.
[0046] Furthermore, after obtaining the current average compression rate for the target memory page, the average compression rate can be compared with the second compression rate that is expected to be achieved. Optionally, if the average compression rate is greater than the second compression rate, it indicates that the average compression rate has not reached the expected second compression rate, that is, the current compression effect on the target memory page has not yet reached the expected compression effect. Therefore, it is possible to further predict whether the expected compression effect can be achieved by continuing to compress the second data with the current compression effect. Specifically, the product of the data volume of the second data in the target memory page and the real-time compression rate can be obtained to obtain the first target data volume. Among them, the first target data volume is the data volume of the second data after the compression process is performed on the second data, which is predicted based on the real-time compression rate of the first data that has completed the compression process.
[0047] Step S232: obtaining a sum of the first target data volume and the data volume of the first data in the target memory page after the compression process is completed, as the second target data volume.
[0048] Step S233: obtaining a ratio of the second target data volume to the data volume of all data in the target memory page before compression as a predicted compression ratio.
[0049] Furthermore, the sum of the first target data volume and the data volume of the first data in the target memory page after the compression process is completed is obtained as the second target data volume. The second target data volume can also be understood as the data volume predicted after the compression process is completed for all the data in the target memory page based on the real-time compression rate of the first data that has currently completed the compression process. Then, after the second target data volume is predicted, the predicted compression rate can be calculated. Specifically, the ratio of the second target data volume to the amount of all data in the target memory page before compression can be obtained as the predicted compression rate. The predicted compression rate can be understood as the final compression rate calculated after the compression process is completed for all the data in the target memory page according to the real-time compression rate of the first data that has currently completed the compression process.
[0050] Step S234: If the predicted compression ratio is greater than the second compression ratio, stop compressing the second data in the target memory page.
[0051] After obtaining the predicted compression rate, the predicted compression rate can be compared with the second compression rate. Optionally, if the predicted compression rate is greater than the second compression rate, it can be determined that even after continuing to completely compress the second data in the target memory page, the final compression rate cannot reach the expected second compression rate, that is, the final overall compression effect is not good. At this time, the compression process of the second data in the target memory page can be stopped. In this way, it helps to avoid the occurrence of problems such as the consumption of computing resources of the device due to continued compression when it is difficult to obtain an effective compression effect; time costs and computing resources of the device are saved.
[0052] Step S235: If the predicted compression rate is less than or equal to the second compression rate, continue to compress the second data in the target memory page.
[0053] Optionally, if the predicted compression rate is less than or equal to the second compression rate, it can be determined that the expected second compression rate, that is, the expected compression effect, can be achieved by continuing to compress the second data in the target memory page according to the real-time compression rate of the first data that has been compressed. At this time, the second data in the target memory page can continue to be compressed to achieve a better compression effect for the target memory page and release more memory space.
[0054] In some embodiments, the expected second compression rate can be a preset value, and can also be automatically adjusted according to actual needs, so as to achieve more flexible data compression. Specifically, before step S230, the current remaining memory space can also be obtained; if the amount of data that can be stored in the remaining memory space is less than or equal to the third target data amount, it can be determined that the current remaining memory space is small, so the current second compression rate can be reduced. Based on this, the average compression rate obtained is compared with the reduced second compression rate. If the average compression rate is greater than the reduced second compression rate, the second data in the target memory page continues to be compressed; if the predicted compression rate is less than or equal to the reduced second compression rate, the second data in the target memory page continues to be compressed. Among them, the third target data amount can be a preset value, of course, it can also be modified according to actual needs, and this embodiment does not limit this. In this way, when the current remaining memory space is insufficient, the reduced second compression rate is automatically reduced, that is, the compression strength of the data in the target memory page is increased, so that the memory space occupied by all data in the target memory page after the compression process is completed is smaller, thereby releasing more memory space.
[0055] Step S240: If the average compression ratio is less than or equal to the second compression ratio, stop compressing the second data in the target memory page, and store the compressed first data and the uncompressed second data in the target memory page.
[0056] Optionally, if the average compression rate is less than or equal to the second compression rate, it can be determined that the average compression rate has reached the desired second compression rate, that is, the current compression effect on the target memory page has reached the desired compression effect. At this time, the compression process on the target memory page can be stopped to achieve the desired compression effect, saving compression time and compression cost. In addition, the compressed first data and the uncompressed second data in the target memory page are stored.
[0057] In other implementations, if the real-time compression ratio is greater than or equal to the first compression ratio, the compression process on the target memory page is stopped.
[0058] In this embodiment, during the compression process of the memory page, the real-time compression rate can be monitored. When the real-time compression rate is high, the compression process is terminated in advance, thereby avoiding the occurrence of problems such as continued compression leading to consumption of computing resources of the device when it is difficult to obtain an effective compression effect; when the real-time compression rate is not high, the average compression rate is further obtained, and based on the average compression rate, it is determined whether it is necessary to continue to compress the target memory page; when the average compression rate reaches the expected second compression rate, the compression process of the target memory page can be stopped to save compression costs and compression time; when the average compression rate does not reach the expected second compression rate, the target memory page is further compressed to achieve a better compression effect and release more memory space. It can be seen that the compression strategy can be dynamically adjusted according to the real-time compression effect in the memory compression process, while ensuring that computing resources are not wasted, the compression efficiency and compression effect are improved as much as possible; at the same time, the stability of the compression efficiency and compression rate is also improved.
[0059] Please refer to Figure 4 , Figure 4 A flowchart of a data compression method provided in another embodiment of the present application is shown below. Figure 4 The data compression method provided in the embodiment of the present application is described in detail. The data compression method may include the following steps:
[0060] Step S310: in the process of compressing the target memory page, obtaining the real-time compression rate of the first data that has completed the compression process in the target memory page.
[0061] In this embodiment, the specific implementation of step S310 can refer to the content of the above-mentioned embodiment, which will not be repeated here.
[0062] Step S320: Obtain the current CPU occupancy rate.
[0063] Step S330: If the CPU occupancy rate is greater than or equal to the target occupancy rate, and the real-time compression rate is greater than or equal to the first compression rate, then stop compressing the target memory page.
[0064] It is understandable that the memory compression process will also occupy certain CPU resources. Therefore, in the process of dynamically adjusting the compression strategy, the CPU occupancy rate can also be combined with the condition of the CPU occupancy rate. Specifically, the current CPU occupancy rate is obtained, and the CPU occupancy rate is compared with the target occupancy rate; optionally, if the CPU occupancy rate is greater than or equal to the target occupancy rate, it can be determined that the CPU resources are relatively scarce. If the real-time compression rate is greater than or equal to the first compression rate at this time, it can be determined that if the target memory page continues to be compressed, the final compression effect is also poor, and it will also cause more CPU resources to be occupied, thereby affecting other processes that occupy CPU resources. Therefore, at this time, the compression process of the target memory page can be stopped. In this way, when the CPU resources are relatively scarce and the real-time compression effect is poor, the compression process of the target memory page is immediately stopped, which not only avoids the occurrence of invalid compression, but also avoids the occurrence of problems such as the compression process preempting the CPU resources when the CPU resources are scarce, thereby affecting the operation of other processes.
[0065] Step S340: If the CPU occupancy rate is less than the target occupancy rate, and the real-time compression rate is greater than or equal to the first compression rate, continue to compress the target memory page.
[0066] Optionally, if the CPU occupancy rate is less than the target occupancy rate, it can be determined that the current CPU resources are relatively idle. Even if the real-time compression rate obtained at this time is greater than or equal to the first compression rate, that is, the real-time compression effect for the current target memory page is not very good, some memory can still be released to a certain extent after compression processing. Therefore, the target memory page can continue to be compressed.
[0067] In other embodiments, if the CPU occupancy rate is greater than or equal to the occupancy rate threshold, even if the real-time compression rate is less than the first compression rate, the compression process of the target memory page is still stopped, wherein the occupancy rate threshold is greater than the aforementioned target occupancy rate. In this way, when the CPU occupancy rate is very high, the memory compression process is stopped as much as possible to avoid affecting the operation of other more important processes.
[0068] In this manner, if the CPU occupancy rate is less than the occupancy rate threshold, and the real-time compression rate is less than the first compression rate, the target memory page can continue to be compressed to achieve a better compression effect and release more memory space. The specific implementation method of continuing to compress the target memory page when the real-time compression rate is less than the first compression rate can be found in the aforementioned embodiment, which will not be described in detail here.
[0069] In this embodiment, the current CPU occupancy rate and the real-time compression rate are combined to comprehensively judge whether to continue to compress the target memory page. If the CPU occupancy rate is greater than or equal to the target occupancy rate and the real-time compression rate is greater than or equal to the first compression rate, the compression process of the target memory page is stopped; if the CPU occupancy rate is less than the target occupancy rate and the real-time compression rate is greater than or equal to the first compression rate, the compression process of the target memory page is continued. In this way, while improving the compression efficiency and compression effect, it also ensures the smooth progress of other processes.
[0070] Please refer to Figure 5 , which shows a structural block diagram of a data compression device 400 provided in an embodiment of the present application. The device 400 may include: a compression rate acquisition module 410 and a compression processing module 420.
[0071] The compression rate acquisition module 410 is used to acquire the real-time compression rate of the first data that has completed the compression process in the target memory page during the process of compressing the target memory page.
[0072] The compression processing module 420 is used to stop compressing the target memory page if the real-time compression rate is greater than or equal to the first compression rate; if the real-time compression rate is less than the first compression rate, continue compressing the target memory page.
[0073] In some embodiments, the compression rate acquisition module 410 can be specifically used to obtain the ratio of the data amount of the first data in the target memory page before compression to the data amount of the first data in the target memory page after the compression process is completed, as the real-time compression rate of the first data.
[0074] In some embodiments, the compression processing module 420 includes: an average compression rate acquisition unit and a compression processing unit. The average compression rate acquisition unit can be used to obtain the current average compression rate for the target memory page if the real-time compression rate is less than the first compression rate. The compression processing unit can be used to continue to compress the second data in the target memory page if the average compression rate is greater than a second compression rate, the second compression rate is less than the first compression rate, and the second data is other data in the target memory page except the first data; if the average compression rate is less than or equal to the second compression rate, stop compressing the second data in the target memory page, and store the compressed first data and the uncompressed second data in the target memory page.
[0075] In this manner, the compression processing unit can be specifically used to: if the average compression rate is greater than the second compression rate, obtain the product of the data volume of the second data in the target memory page and the real-time compression rate to obtain the first target data volume; obtain the sum of the first target data volume and the data volume of the first data in the target memory page after the compression processing is completed, as the second target data volume; obtain the ratio of the second target data volume to the data volume of all data in the target memory page before compression, as the predicted compression rate; if the predicted compression rate is greater than the second compression rate, stop compressing the second data in the target memory page; if the predicted compression rate is less than or equal to the second compression rate, continue compressing the second data in the target memory page.
[0076] In this manner, the average compression ratio acquisition unit can be specifically used to obtain the ratio between the amount of data of all data in the target memory page after compression processing of the first data is completed and the amount of data of all data in the target memory page before compression, as the average compression ratio.
[0077] In some embodiments, the data compression device 400 may further include: a memory space acquisition module and a compression rate adjustment module. The memory space acquisition module may be used to acquire the current remaining memory space before continuing to compress the second data in the target memory page if the average compression rate is greater than the second compression rate. The compression rate adjustment module may be used to reduce the second compression rate if the amount of data that can be stored in the remaining memory space is less than or equal to the third target data amount. The compression processing unit may be specifically used to continue to compress the second data in the target memory page if the average compression rate is greater than the reduced second compression rate.
[0078] In some embodiments, the data compression device 400 may further include: a CPU occupancy acquisition module. The CPU occupancy acquisition module may be used to acquire the current CPU occupancy of the central processing unit before stopping the compression processing of the target memory page if the real-time compression rate is greater than or equal to the first compression rate. The compression processing module 420 may be specifically used to stop the compression processing of the target memory page if the CPU occupancy is greater than or equal to the target occupancy rate and the real-time compression rate is greater than or equal to the first compression rate; if the CPU occupancy is less than the target occupancy rate and the real-time compression rate is greater than or equal to the first compression rate, continue to compress the target memory page.
[0079] In some implementations, the compression rate acquisition module 410 may be specifically used to acquire the real-time compression rate of the compressed first data in the target memory page each time a fourth target amount of data is compressed during the process of compressing the target memory page.
[0080] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.
[0081] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.
[0082] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.
[0083] In summary, during the compression of memory pages, the real-time compression rate can be monitored. When the real-time compression rate is high, the compression process is terminated in advance, avoiding the occurrence of problems such as continued compression when it is difficult to obtain an effective compression effect, which leads to the consumption of computing resources of the device; when the real-time compression rate is not high, the target memory page is tried to be compressed to achieve a better compression effect and release more memory space. It can be seen that the real-time compression effect can be obtained according to the real-time compression rate, and the compression strategy can be dynamically adjusted to ensure that computing resources are not wasted while improving the compression efficiency and compression effect as much as possible.
[0084] The following will be combined Figure 6 A computer device provided by the present application is described.
[0085] Reference Figure 6 , Figure 6The structural block diagram of a computer device 500 provided in an embodiment of the present application is shown, and the above method provided in an embodiment of the present application can be executed by the computer device 500. The computer device can be an electronic terminal with a data processing function, and the electronic terminal includes but is not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3) player and an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) player, etc.; of course, the computer device can also be a server, and the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
[0086] The computer device 500 in the embodiment of the present application may include one or more of the following components: a processor 501, a memory 502, and one or more applications, wherein the one or more applications may be stored in the memory 502 and configured to be executed by one or more processors 501, and the one or more programs are configured to execute the method as described in the aforementioned method embodiment.
[0087] The processor 501 may include one or more processing cores. The processor 501 uses various interfaces and lines to connect various parts of the entire computer device 500, and executes various functions and processes data of the computer device 500 by running or executing instructions, programs, code sets or instruction sets stored in the memory 502, and calling data stored in the memory 502. Optionally, the processor 501 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 501 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem can also be integrated into the processor 501 and implemented separately through a communication chip.
[0088] The memory 502 may include a random access memory (RAM) or a read-only memory (ROM). The memory 502 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as the various corresponding relationships described above) created by the computer device 500 during use.
[0089] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.
[0090] In several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or modules, which may be electrical, mechanical or other forms.
[0091] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.
[0092] Please refer to Figure 7 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 600 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.
[0093] The computer readable storage medium 600 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer readable storage medium 600 has storage space for program code 610 that performs any method step of the above method. These program codes can be read from or written to one or more computer program products. The program code 610 can be compressed, for example, in an appropriate form.
[0094] In some embodiments, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the steps in the above-mentioned method embodiments.
[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A data compression method, characterized in that: The method comprises: In the process of compressing the target memory page, obtaining a real-time compression rate of the first data that has completed the compression processing in the target memory page; If the real-time compression rate is greater than or equal to the first compression rate, stopping the compression process on the target memory page; If the real-time compression rate is less than the first compression rate, the target memory page continues to be compressed.
2. The method according to claim 1, characterized in that The obtaining of the real-time compression rate of the first data in the target memory page that has completed the compression process includes: A ratio of the amount of the first data in the target memory page before compression to the amount of the first data in the target memory page after compression is completed is obtained as a real-time compression rate of the first data.
3. The method according to claim 1, characterized in that If the real-time compression rate is less than the first compression rate, continuing to compress the target memory page includes: If the real-time compression rate is less than the first compression rate, obtaining the current average compression rate for the target memory page; If the average compression ratio is greater than a second compression ratio, continue to compress the second data in the target memory page, the second compression ratio is less than the first compression ratio, and the second data is other data in the target memory page except the first data; If the average compression rate is less than or equal to the second compression rate, the compression process on the second data in the target memory page is stopped, and the compressed first data and the uncompressed second data in the target memory page are stored.
4. The method according to claim 3, characterized in that If the average compression ratio is greater than the second compression ratio, continuing to compress the second data in the target memory page includes: If the average compression rate is greater than the second compression rate, obtaining the product of the data volume of the second data in the target memory page and the real-time compression rate to obtain a first target data volume; Obtaining a sum of the first target data volume and the data volume of the first data in the target memory page after the compression process is completed as the second target data volume; Obtaining a ratio of the second target data volume to the data volume of all data in the target memory page before compression as a predicted compression ratio; If the predicted compression rate is greater than the second compression rate, stopping the compression process on the second data in the target memory page; If the predicted compression rate is less than or equal to the second compression rate, continue to compress the second data in the target memory page.
5. The method according to claim 3, characterized in that: The obtaining the current average compression rate for the target memory page includes: A ratio between the amount of all data in the target memory page after the compression process for the first data is completed and the amount of all data in the target memory page before compression is obtained as the average compression ratio.
6. The method according to claim 3, characterized in that Before continuing to compress the second data in the target memory page if the average compression ratio is greater than the second compression ratio, the method further includes: Get the current remaining memory space; If the amount of data that can be stored in the remaining memory space is less than or equal to the third target data amount, reducing the second compression ratio; If the average compression ratio is greater than the second compression ratio, continuing to compress the second data in the target memory page includes: If the average compression ratio is greater than the reduced second compression ratio, the second data in the target memory page continues to be compressed.
7. The method according to any one of claims 1 to 6, characterized in that: Before stopping the compression process on the target memory page if the real-time compression rate is greater than or equal to the first compression rate, the method further includes: Get the current CPU usage; If the real-time compression rate is greater than or equal to the first compression rate, stopping the compression process on the target memory page includes: If the CPU occupancy rate is greater than or equal to the target occupancy rate, and the real-time compression rate is greater than or equal to the first compression rate, the compression process on the target memory page is stopped.
8. The method according to claim 7, characterized in that After obtaining the current CPU occupancy rate, the method further includes: If the CPU occupancy rate is less than the target occupancy rate, and the real-time compression rate is greater than or equal to the first compression rate, the target memory page continues to be compressed.
9. The method according to any one of claims 1 to 6, characterized in that: In the process of compressing the target memory page, obtaining the real-time compression rate of the first data that has completed the compression processing in the target memory page includes: In the process of compressing the target memory page, each time data of a fourth target data amount is compressed, a real-time compression rate of the first data that has completed the compression process in the target memory page is obtained.
10. A data compression device, characterized in that: The device comprises: A compression rate acquisition module, used for acquiring a real-time compression rate of the first data that has completed the compression process in the target memory page during the process of compressing the target memory page; The compression processing module is used to stop compressing the target memory page if the real-time compression rate is greater than or equal to the first compression rate; if the real-time compression rate is less than the first compression rate, continue compressing the target memory page.
11. A computer device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, and the program code can be called by a processor to execute the method according to any one of claims 1 to 9.
