Memory compression method and device, equipment, storage medium and product

Abstract

The invention discloses a memory compression method and device, equipment, a storage medium and a product. Firstly, memory use information and data attributes of a data block are acquired; wherein the data attribute comprises at least one of a data type, a current access frequency and an importance index; then, selecting a target compression strategy for the data block according to the memory use information and the data attribute; and finally, the target compression strategy is utilized to perform targeted compression on the data block, so that the memory compression effect is improved, and memory resources can be effectively utilized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a memory compression method, device, equipment, storage medium and product. Background Art

[0002] As the software systems of electronic devices such as smartphones and tablets continue to develop towards intelligence, the applications of electronic devices are becoming increasingly complex, and memory management is needed to meet the running requirements of applications. At present, in terms of memory management, although compression algorithms are used to reduce the occupation of physical memory, they are often limited by fixed compression strategies. This "one-size-fits-all" compression method lacks targeted optimization for different data, resulting in poor compression effects and failure to effectively utilize memory resources. Summary of the invention

[0003] Based on this, the present invention provides a memory compression method, device, equipment, storage medium and product, which can determine the target compression strategy according to the memory usage information and data attributes of the database, perform memory compression in a targeted manner, improve the memory compression effect, and enable memory resources to be effectively utilized.

[0004] To achieve the above object, an embodiment of the present invention provides a memory compression method, comprising:

[0005] Acquire system memory usage information and data attributes of data blocks; wherein the data attributes include at least one of data type, current access frequency, and importance index;

[0006] Determine a target compression strategy according to the memory usage information and the data attributes;

[0007] The data block is compressed according to the target compression strategy.

[0008] To achieve the above object, an embodiment of the present invention further provides a memory compression device, comprising:

[0009] A data acquisition module, used to acquire system memory usage information and data attributes of data blocks; wherein the data attributes include at least one of data type, current access frequency and importance index;

[0010] A compression strategy determination module, used to determine a target compression strategy according to the memory usage information and the data attributes;

[0011] A compression module is used to compress the data block according to the target compression strategy.

[0012] To achieve the above objectives, an embodiment of the present invention also provides a memory compression device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the memory compression method described in any of the above embodiments.

[0013] To achieve the above objectives, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the memory compression method as described in any of the above embodiments.

[0014] To achieve the above objectives, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the memory compression method as described in any of the above embodiments.

[0015] Compared with the prior art, the memory compression method, apparatus, device, storage medium and product disclosed in the embodiments of the present invention first obtain memory usage information and data attributes of a data block; wherein the data attributes include at least one of a data type, a current access frequency and an importance index; then, a target compression strategy is selected for the data block based on the memory usage information and the data attributes; finally, the data block is compressed in a targeted manner using the target compression strategy, thereby improving the memory compression effect and enabling memory resources to be effectively utilized. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the implementation mode will be briefly introduced below. Obviously, the drawings described below are only some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of a memory compression method provided by an embodiment of the present invention;

[0018] Figure 2 It is a structural schematic diagram of a memory compression device provided by an embodiment of the present invention;

[0019] Figure 3 It is a structural schematic diagram of a memory compression device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1 , is a flow chart of a memory compression method provided by an embodiment of the present invention. Specifically, the memory compression method comprises steps S1 to S3:

[0022] S1. Obtaining system memory usage information and data attributes of a data block; wherein the data attributes include at least one of a data type, a current access frequency, and an importance index;

[0023] S2. determining a target compression strategy according to the memory usage information and the data attributes;

[0024] S3. Compress the data block according to the target compression strategy.

[0025] It is understandable that in memory management, memory pages that are not currently used by any system process or application can be appropriately compressed to release main memory for use by other processes, thereby achieving the purpose of improving system performance. The method can be seamlessly integrated with existing operating systems and applications, and the memory management process is transparent to users and applications.

[0026] Specifically, real-time monitoring of the system's memory usage, recording memory allocation and release, data access frequency, etc., monitors the usage of the central processing unit (CPU), memory, and computer interface (Input / Output, I / O). For example: through the application programming interface (Application Programming Interface, API) or monitoring tools provided by the operating system, regularly collect memory usage, including total memory, available memory, used memory, cache memory, track the memory allocation and release operations of each application or process, and record the time points and amounts of memory usage growth and decrease; by analyzing memory access logs or using memory access monitoring tools, count the access frequency of each memory block (memory blocks are used to store data blocks), identify high-frequency access and low-frequency access data, and analyze memory access timing data through machine learning algorithms or pattern recognition technology to identify access patterns, such as sequential access and random access; through the API or monitoring tools provided by the operating system, Regularly collect CPU usage, including overall CPU usage, each core usage, and CPU usage of each process, analyze CPU load, identify high-load and low-load periods, and evaluate CPU usage efficiency and bottlenecks; regularly collect I / O usage, including disk I / O and network I / O, through the API or monitoring tools provided by the operating system, record the frequency and data volume of read and write operations, analyze I / O load, identify high-load and low-load periods, and evaluate the performance and bottlenecks of the I / O subsystem; integrate the data of memory, CPU, and I / O usage, conduct comprehensive analysis, and identify system performance bottlenecks and resource waste points. Based on the analysis results, design and apply intelligent memory management strategies, including dynamically adjusting memory allocation and release, optimizing data compression and decompression strategies, adjusting process priorities, etc., to ensure efficient use of system resources. For example, based on the data obtained from monitoring, the system's memory usage information and data attributes of the data block are determined. The memory usage information may include information such as total memory, available memory, and used memory. The data attributes may include data type, current access frequency, and importance index. Based on the correspondence between the pre-set memory usage information, data attributes, and compression strategy, a target compression strategy is determined for the data block, and the data block is compressed according to the target compression strategy. Among them, the importance index can be pre-defined for each data block according to a pre-set rule, or the importance can be determined based on CPU usage and I / O usage, etc. For example, the importance index value set for a data block that needs to frequently call CPU resources and I / O resources is relatively large.

[0027] Compared with the prior art, this embodiment provides a memory compression method, which first obtains the system's memory usage information and data attributes of a data block; wherein the data attributes include at least one of a data type, a current access frequency, and an importance index; then, a target compression strategy is selected for the data block based on the memory usage information and the data attributes; finally, the data block is compressed in a targeted manner using the target compression strategy, thereby improving the memory compression effect and enabling memory resources to be effectively utilized.

[0028] In a preferred implementation, determining a target compression strategy according to the memory usage information and the data attributes includes:

[0029] Determine the current memory pressure based on the current memory usage information;

[0030] Based on the corresponding relationship between the attribute and the limiting parameter, determining the compression rate limiting parameter according to the data attribute;

[0031] Determine memory pressure limit parameters based on memory usage information in recent historical periods;

[0032] Calculate the target compression rate according to the compression rate limit parameter, the memory pressure limit parameter and the current memory pressure; wherein the greater the current memory pressure, the greater the target compression rate;

[0033] A target compression algorithm and compression parameters thereof are determined according to the target compression rate and the data type of the data block.

[0034] It is worth noting that based on the use of statistical characteristics and repetitive patterns of data, compression algorithms can be divided into lossless compression and lossy compression. Common lossless compression algorithms include Huffman coding, dictionary coding, and predictive coding, while lossy compression algorithms include conversion and quantization. The classification of compression algorithms is not limited to the above specific types, and may also include other types.

[0035] Specifically, the current memory pressure of the system is determined according to the real-time monitored memory usage information of the system, such as the memory utilization rate or memory usage of the system (the greater the memory utilization rate / memory usage, the greater the current memory pressure). In addition, according to the correspondence between the pre-set attributes and the limiting parameters, the compression rate limiting parameters are determined according to the data attributes of the data block, wherein the compression rate limiting parameters are determined according to the data attributes of the data block. Taking the data attributes including the data type as an example, different data types (such as pictures, texts, etc.) have different requirements for compression rates when compressing. For information-rich data, too large a compression rate may cause data information loss. Therefore, the compression rate limiting parameters can be determined according to the data attributes, and the compression rate limiting parameters are used to assist in calculating the target compression rate. It can be understood that the data attributes can also include the current access frequency and importance index, etc., which are not limited here. For example, the larger the importance index, the smaller the target compression rate; the larger the current access frequency, the smaller the target compression rate, which is convenient for the rapid decompression of subsequent data blocks. In addition, the system's memory pressure range (minimum memory pressure and maximum memory pressure) will also have a certain impact on the target compression rate of the data block. Therefore, the memory pressure range is applied to the calculation process of the target compression rate. After determining the target compression rate, the target compression algorithm and its compression parameters are determined in combination with the data type for compressing the data block. It is understandable that different compression algorithms can achieve different compression rates, and different data types (such as text, pictures, etc.) have different restrictions on compression rates and requirements for compression parameters. Therefore, the corresponding target compression algorithm and its compression parameters are selected according to the target compression rate and data type.

[0036] This implementation can automatically adjust the compression rate according to the current memory pressure of the system, the memory pressure limit parameter and the compression rate limit parameter of the data block, thereby maximizing the memory utilization while ensuring the system performance.

[0037] In a preferred embodiment, the compression rate limiting parameter includes at least one of a minimum compression rate and a maximum compression rate, and the memory pressure limiting parameter includes at least one of a minimum memory pressure and a maximum memory pressure.

[0038] In a preferred embodiment, the compression rate limiting parameter includes a minimum compression rate and a maximum compression rate, and the memory pressure limiting parameter includes a minimum memory pressure and a maximum memory pressure;

[0039] The calculating the target compression rate according to the compression rate limit parameter, the memory pressure limit parameter and the current memory pressure includes:

[0040] Subtract the minimum compression rate from the maximum compression rate to obtain a compression coefficient;

[0041] Divide the difference between the current memory pressure and the minimum memory pressure by the difference between the maximum memory pressure and the minimum memory pressure to obtain a pressure ratio;

[0042] A target compression ratio is calculated according to the minimum compression ratio, the compression coefficient and the pressure ratio; wherein the target compression ratio is positively correlated with the minimum compression ratio, the compression coefficient and the pressure ratio respectively.

[0043] Specifically, the compression coefficient is calculated by the maximum compression rate and the minimum compression rate, and the pressure ratio is calculated by the system's current memory pressure, the lowest memory pressure and the highest memory pressure. The target compression rate is determined by adding the value calculated based on the compression coefficient and the compression ratio to the minimum compression rate.

[0044] Further, the calculating the target compression ratio according to the minimum compression ratio, the compression coefficient and the pressure ratio includes:

[0045] The pressure ratio is multiplied by a set adjustment parameter to obtain a pressure index; wherein the set adjustment parameter is a negative number;

[0046] The target compression ratio is obtained by subtracting the pressure exponent of the natural constant from one and multiplying the result by the compression coefficient and adding the minimum compression ratio.

[0047] Specifically, the target compression ratio is calculated as follows:

[0048]

[0049] Among them, C(t) is the compression rate at time t (i.e., the target compression rate), C min is the minimum compression ratio, C max is the maximum compression ratio, P(t) is the memory pressure at time t, and P min is the minimum memory pressure, P max is the maximum memory pressure, and -α is the tuning parameter.

[0050] It is worth noting that the calculation method of the target compression ratio is not limited to the above specific formula. The formula can be set according to the actual situation, and it is only necessary to ensure that the target compression ratio is positively correlated with the minimum compression ratio, the compression coefficient and the pressure ratio. It is worth noting that the target compression ratio refers to 1 minus the ratio of the expected compressed data block size to the data block size before compression.

[0051] In one implementation, the data type is obtained by: extracting characteristic information of the data block, and determining the data type of the data block according to the characteristic information of the data block;

[0052] The determining of the memory pressure limit parameter according to the memory usage information of the recent historical period includes: determining the historical memory pressure of the recent historical period according to the memory usage information of the recent historical period;

[0053] A minimum memory pressure and a maximum memory pressure are selected from the historical memory pressures in the recent historical period, and the minimum memory pressure and the maximum memory pressure are used as memory pressure limit parameters.

[0054] Specifically, the determination of the data type and the setting of the target compression algorithm and its compression parameters are as follows:

[0055] Extract relevant feature information from the data block, the formula is: F(D) = {f1(D), f2(D), ... f i (D),…,f n (D)}; where D is the data block, F(D) is the feature set, and f i (D) is the i-th feature extraction function;

[0056] According to the extracted feature information, the data is classified into different types, the formula is: C(F(D)) = data type; where C is the classification model function, F(D) is the feature set, and the classification result is the data type;

[0057] According to the feature information, classification results and target compression rate, the parameters of the compression algorithm are selected. The formula is: P(C(F(D)), F(D)) = parameters of the compression algorithm; where P is a parameter selection function, the classification results and feature sets are input, and the output is the parameters of the compression algorithm. It can be understood that for different data types, when compressing data blocks, different parameters of the compression algorithm are set, so that the data blocks are compressed as much as possible without information loss;

[0058] Use the selected compression algorithm and its parameters to compress the data. The formula is: Z(D,A,

[0059] P)=D compressed ; The output is a compressed data block.

[0060] Specifically, the memory pressure limit parameter is determined as follows: memory usage information of a recent historical period is obtained to determine the historical memory pressure of the period, and the maximum and minimum values ​​are selected from the historical memory pressure of the period as the memory pressure limit parameter. It is worth noting that the specific length of the recent historical period can be set according to actual conditions and is not limited here.

[0061] In this embodiment, the concept of content-aware compression is introduced. By combining the data type, access frequency, importance and system memory pressure to select the most suitable compression algorithm and its parameters for compression, the compression efficiency is significantly improved while ensuring the integrity and accessibility of the data.

[0062] In one embodiment, the method further comprises:

[0063] Acquire data interaction characteristics of the data block; wherein the data interaction characteristics include memory usage information of the system in a recent first set time length and access frequency of the data block in a recent second set time length;

[0064] Inputting the data interaction features into a pre-trained prediction model to obtain a memory usage prediction value and an access frequency prediction value;

[0065] Designing a cache strategy according to the memory usage prediction value and the access frequency prediction value;

[0066] The data in the data block is decompressed and cached according to the cache strategy.

[0067] Specifically, the trained prediction model is used to predict the future memory usage (i.e., the memory usage prediction value), and the formula is: u(t) = M trained (F(t)); where u(t) is the predicted memory usage at time t, and F(t) is the data interaction feature of the data block at time t. It is worth noting that the data interaction feature includes the access frequency of the data block in the most recent first set time and the memory usage information of the system in the most recent second set time. It is understandable that the specific durations of the first set time and the second set time can be equal or different, and can be set according to actual conditions, and are not limited here.

[0068] Use the trained prediction model to predict the future data access frequency of the data block (i.e., the access frequency prediction value), the formula is: a(t) = M trained (F(t)); where a(t) is the predicted value of the access frequency at time t, and F(t) is the characteristic information of the database at time t.

[0069] According to the predicted memory usage and the predicted data access frequency, an intelligent decompression cache strategy is designed. The formula is: K(u(t), a(t)) = cache strategy; where K is the cache strategy function, u(t) is the predicted memory usage, a(t) is the predicted access frequency, and the output is the cache strategy;

[0070] According to the cache strategy, the decompressed data of frequently accessed data blocks are cached. The formula is: Cache(D,

[0071] K)=Dcached ; Among them, Cache is the cache function, D is the data block, K is the cache strategy, and the output is the cached decompressed data.

[0072] This implementation can accurately predict future memory usage trends and data access frequencies based on known data, allowing the system to make optimization adjustments in advance, ensuring efficient use of resources and improving user experience.

[0073] In one implementation, the larger the predicted value of memory usage is, the shorter the cache duration in the cache strategy is; and the larger the predicted value of access frequency is, the longer the cache duration in the cache strategy is.

[0074] Exemplarily, in order to enable the system to process data as quickly as possible while having sufficient memory resources for information processing, this embodiment analyzes the future memory usage trend and data access frequency of the data block, and caches the decompressed data of the frequently accessed data block in advance, thereby significantly reducing the decompression time during data access, improving the overall performance and response speed of the system, and also increasing the data cache time of the frequently accessed data block, reducing the compression and decompression frequency of the data block, and reducing the waste of computing resources. This is particularly important for application scenarios that require fast response and high availability (such as financial transactions, online games, etc.), and can bring better user experience and higher customer satisfaction. In the case where the memory usage prediction value of the data block is too large, if the cache time of the data block is too long, it will cause the cache space to be unable to meet the cache requirements of other data for a long time, reducing the data processing efficiency of the system, so it is necessary to shorten the cache time with a large memory usage prediction value.

[0075] It is worth noting that the design method of the cache strategy is not limited to the above specific examples, and can be reasonably designed according to the memory usage prediction value and the access frequency prediction value, and its specific design method is not limited here.

[0076] In one embodiment, the prediction model is trained in the following manner:

[0077] Acquire a number of training samples; wherein the training samples at least include historical memory usage information of the system and historical access frequencies of known data blocks;

[0078] The training samples are used to train the machine learning model to obtain a prediction model.

[0079] Specifically, we use machine learning models to predict future memory usage trends and data access patterns, design an intelligent decompression cache mechanism, and cache decompressed data for frequently accessed data blocks. The training method of the machine learning model is as follows: Use historical data to train the model, the formula is: M trained= Train(M, F'(D)); Train is the training function, M is the initial machine learning model, F'(D) is the training sample, M trained is the trained model (i.e., prediction model).

[0080] Exemplarily, the training samples include historical memory usage information and historical access frequency of several data blocks. The training, application and update process of the prediction model is as follows: 1. Through the API or monitoring tools provided by the operating system, regularly collect data such as system memory usage and system load in different time periods. The data may include timestamps, memory usage (i.e. historical memory usage information), CPU usage, IO operations, network usage, historical access frequency of data blocks, memory allocation and release, total system memory, cache memory, available memory, etc.; 2. Monitor the system's memory usage and system load in real time, and use tools such as Prometheus, Grafana, Nagios, etc. for data collection and monitoring; 3. Select appropriate machine learning models according to data characteristics, such as linear regression, decision tree, random forest, LST M, etc., use the historical data in step 1 to train the model, evaluate the performance of the model, use cross-validation and other techniques to adjust the model parameters, improve the model performance, and use techniques such as grid search and random search to tune hyperparameters; 5. Deploy the trained model to the production environment, and use frameworks such as Flask and Django to provide prediction API services; 6. Input the data collected in step 2 into the prediction model, output the prediction results of the prediction model, and analyze the trends of memory usage and system load (i.e., memory usage prediction value and access frequency prediction value) based on the prediction results. 7. Regularly retrain the model with the latest historical data to ensure the accuracy and timeliness of the model, and update the model in real time through online learning algorithms such as incremental learning. Among them, Prometheus is an open source monitoring and alarm system that is widely used in various scenarios; Grafana is an open source interactive data visualization platform; Nagios is an open source network monitoring tool.

[0081] In one embodiment, the method further comprises:

[0082] The actual compression rate of the data block is calculated according to the data volume before compression and the data volume after compression of the data block.

[0083] Specifically, after memory compression, the compressed data and related metadata are saved, and the compression effect is verified. The formula is as follows: V(D,D compressed ) = compression effect; where V is the verification function, D is the data block before compression, and D compressedIt is a compressed data block. The output is an evaluation result of the compression effect. The compression effect can be an actual compression ratio. The actual compression ratio = 1 - the amount of data before the data block is compressed / the amount of data after the data block is compressed.

[0084] Compared with the prior art, an embodiment of the present invention provides a memory compression method, which first obtains memory usage information and data attributes of a data block; wherein the data attributes include at least one of a data type, a current access frequency, and an importance index; then, a target compression strategy is selected for the data block based on the memory usage information and the data attributes; finally, the data block is compressed in a targeted manner using the target compression strategy, thereby improving the memory compression effect and enabling memory resources to be effectively utilized.

[0085] See also Figure 2 , Figure 2 A memory compression device provided by an embodiment of the present invention comprises:

[0086] The data acquisition module 21 is used to acquire the system's memory usage information and data attributes of the data block; wherein the data attributes include at least one of the data type, current access frequency and importance index;

[0087] A compression strategy determination module 22, configured to determine a target compression strategy according to the memory usage information and the data attributes;

[0088] The compression module 23 is used to compress the data block according to the target compression strategy.

[0089] In one implementation, the compression strategy determination module 22 is specifically configured to:

[0090] Determine the current memory pressure based on the current memory usage information;

[0091] Based on the corresponding relationship between the attribute and the limiting parameter, determining the compression rate limiting parameter according to the data attribute;

[0092] Determine memory pressure limit parameters based on memory usage information in recent historical periods;

[0093] Calculate a target compression rate according to the compression rate limit parameter, the memory pressure limit parameter and the current memory pressure;

[0094] Based on the correspondence between the compression rate and the compression algorithm, a target compression algorithm is determined according to the target compression rate; wherein the greater the current memory pressure, the greater the target compression rate;

[0095] A target compression algorithm and compression parameters thereof are determined according to the target compression rate and the data type of the data block.

[0096] In one embodiment, the compression rate limiting parameter includes at least one of a minimum compression rate and a maximum compression rate, and the memory pressure limiting parameter includes at least one of a minimum memory pressure and a maximum memory pressure.

[0097] In one implementation, the calculating the target compression rate according to the compression rate limit parameter, the memory pressure limit parameter and the current memory pressure includes:

[0098] Subtracting the minimum compression rate from the maximum compression rate to obtain a compression coefficient; wherein the compression rate limiting parameter includes the minimum compression rate and the maximum compression rate;

[0099] The difference between the current memory pressure and the lowest memory pressure is divided by the difference between the highest memory pressure and the lowest memory pressure to obtain a pressure ratio; wherein the memory pressure limit parameter includes the lowest memory pressure and the highest memory pressure;

[0100] A target compression ratio is calculated according to the minimum compression ratio, the compression coefficient and the pressure ratio; wherein the target compression ratio is positively correlated with the minimum compression ratio, the compression coefficient and the pressure ratio respectively.

[0101] In one embodiment, the calculating the target compression ratio according to the minimum compression ratio, the compression coefficient and the pressure ratio includes:

[0102] The pressure ratio is multiplied by a set adjustment parameter to obtain a pressure index; wherein the set adjustment parameter is a negative number;

[0103] The target compression ratio is obtained by subtracting the pressure exponent of the natural constant from one and multiplying the result by the compression coefficient and adding the minimum compression ratio.

[0104] In one implementation, the data type is obtained by: extracting characteristic information of the data block, and determining the data type of the data block according to the characteristic information of the data block;

[0105] Determining the memory pressure limit parameter according to the memory usage information in the recent historical period includes:

[0106] Determine the historical memory pressure of the recent historical period according to the memory usage information of the recent historical period;

[0107] A minimum memory pressure and a maximum memory pressure are selected from the historical memory pressures in the recent historical period, and the minimum memory pressure and the maximum memory pressure are used as memory pressure limit parameters.

[0108] In one embodiment, the device further includes a decompression cache module, configured to:

[0109] Acquire data interaction characteristics of the data block; wherein, memory usage information of the system in the most recent first set time period and access frequency of the data block in the most recent second set time period;

[0110] Inputting the data interaction features into a pre-trained prediction model to obtain a memory usage prediction value and an access frequency prediction value;

[0111] Designing a cache strategy according to the memory usage prediction value and the access frequency prediction value;

[0112] The data block is decompressed and cached according to the cache strategy.

[0113] In one implementation, the larger the predicted value of memory usage is, the shorter the cache duration in the cache strategy is; and the larger the predicted value of access frequency is, the longer the cache duration in the cache strategy is.

[0114] In one embodiment, the device further comprises a model training module, which is used to:

[0115] Acquire a number of training samples; wherein the training samples at least include historical memory usage information of the system and historical access frequencies of known data blocks;

[0116] The training samples are used to train the machine learning model to obtain a prediction model.

[0117] In one embodiment, the device further includes a compression ratio calculation module, which is used to:

[0118] The compression ratio of the data block is calculated according to the data volume before compression and the data volume after compression of the data block.

[0119] It is worth noting that the specific working process of the memory compression device can refer to the working process of the memory compression method described in the above embodiment, which will not be repeated here.

[0120] Compared with the prior art, the memory compression device disclosed in the embodiment of the present invention first obtains memory usage information and data attributes of a data block; wherein the data attributes include at least one of a data type, a current access frequency, and an importance index; then, a target compression strategy is selected for the data block based on the memory usage information and the data attributes; finally, the data block is compressed in a targeted manner using the target compression strategy, thereby improving the memory compression effect and enabling memory resources to be effectively utilized.

[0121] See also Figure 3 , Figure 3is a schematic diagram of the structure of a memory compression device provided by an embodiment of the present invention. The embodiment of the present invention also provides a memory compression device, including a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31. When the processor 31 executes the computer program, the steps in the above-mentioned memory compression method embodiment are implemented, for example Figure 1 or, the processor 31 implements the functions of each module in the above-mentioned device embodiments when executing the computer program.

[0122] Exemplarily, the computer program may be divided into one or more modules, and the one or more modules are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the memory compression device. For example, the computer program may be divided into multiple modules, and the specific working process of each module may refer to the working process of the memory compression device described in the above embodiment, which will not be repeated here.

[0123] The memory compression device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The memory compression device may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will appreciate that the memory compression device may also include an input / output device, a network access device, a bus, etc.

[0124] The processor 31 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 31 is the control center of the memory compression device, and uses various interfaces and lines to connect various parts of the entire memory compression device.

[0125] The memory 32 can be used to store the computer program and / or module. The processor 31 realizes various functions of the memory compression device by running or executing the computer program and / or module stored in the memory 32 and calling the data stored in the memory 32. The memory 32 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory 32 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0126] Wherein, if the module integrated in the memory compression device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 31, the steps of the above-mentioned various method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory 32, read-only memory 32 (ROM, Read-Only Memory), random access memory 32 (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium.

[0127] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the memory compression method described in any of the above embodiments when executed by the processor 31.

[0128] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A memory compression method, characterized in that: include: Acquire system memory usage information and data attributes of data blocks; wherein the data attributes include at least one of data type, current access frequency, and importance index; Determine a target compression strategy according to the memory usage information and the data attributes; The data block is compressed according to the target compression strategy.

2. The memory compression method according to claim 1, characterized in that: The determining a target compression strategy according to the memory usage information and the data attributes includes: Determine the current memory pressure based on the current memory usage information; Based on the corresponding relationship between the attribute and the limiting parameter, determining the compression rate limiting parameter according to the data attribute; Determine memory pressure limit parameters based on memory usage information in recent historical periods; Calculate the target compression rate according to the compression rate limit parameter, the memory pressure limit parameter and the current memory pressure; wherein the greater the current memory pressure, the greater the target compression rate; A target compression algorithm and compression parameters thereof are determined according to the target compression rate and the data type of the data block.

3. The memory compression method according to claim 2, characterized in that: The compression rate limiting parameter includes at least one of a minimum compression rate and a maximum compression rate, and the memory pressure limiting parameter includes at least one of a minimum memory pressure and a maximum memory pressure.

4. The memory compression method according to claim 2, characterized in that: The calculating the target compression rate according to the compression rate limit parameter, the memory pressure limit parameter and the current memory pressure includes: Subtracting the minimum compression rate from the maximum compression rate to obtain a compression coefficient; wherein the compression rate limiting parameter includes the minimum compression rate and the maximum compression rate; The difference between the current memory pressure and the lowest memory pressure is divided by the difference between the highest memory pressure and the lowest memory pressure to obtain a pressure ratio; wherein the memory pressure limit parameter includes the lowest memory pressure and the highest memory pressure; A target compression ratio is calculated according to the minimum compression ratio, the compression coefficient and the pressure ratio; wherein the target compression ratio is positively correlated with the minimum compression ratio, the compression coefficient and the pressure ratio respectively.

5. The memory compression method according to claim 4, characterized in that: The calculating the target compression ratio according to the minimum compression ratio, the compression coefficient and the pressure ratio includes: The pressure ratio is multiplied by a set adjustment parameter to obtain a pressure index; wherein the set adjustment parameter is a negative number; The target compression ratio is obtained by subtracting the pressure exponent of the natural constant from one and multiplying the result by the compression coefficient and adding the minimum compression ratio.

6. The memory compression method according to claim 2, characterized in that: The data type is obtained by: extracting characteristic information of the data block, and determining the data type of the data block according to the characteristic information of the data block; Determining the memory pressure limit parameter according to the memory usage information in the recent historical period includes: Determine the historical memory pressure of the recent historical period according to the memory usage information of the recent historical period; A minimum memory pressure and a maximum memory pressure are selected from the historical memory pressures in the recent historical period, and the minimum memory pressure and the maximum memory pressure are used as memory pressure limit parameters.

7. The memory compression method according to claim 1, characterized in that: Also includes: Acquire data interaction characteristics of the data block; wherein the data interaction characteristics include memory usage information of the system in a recent first set time length and access frequency of the data block in a recent second set time length; Inputting the data interaction features into a pre-trained prediction model to obtain a memory usage prediction value and an access frequency prediction value; Designing a cache strategy according to the memory usage prediction value and the access frequency prediction value; The data block is decompressed and cached according to the cache strategy.

8. The memory compression method according to claim 7, characterized in that: The larger the predicted value of memory usage is, the shorter the cache duration in the cache strategy is; the larger the predicted value of access frequency is, the longer the cache duration in the cache strategy is.

9. The memory compression method according to claim 7, characterized in that: The prediction model is trained in the following way: Acquire a number of training samples; wherein the training samples at least include historical memory usage information of the system and historical access frequencies of known data blocks; The training samples are used to train the machine learning model to obtain a prediction model.

10. The memory compression method according to claim 1, characterized in that: Also includes: The actual compression rate of the data block is calculated according to the data volume before compression and the data volume after compression of the data block.

11. A memory compression device, characterized in that: include: A data acquisition module, used to acquire system memory usage information and data attributes of data blocks; wherein the data attributes include at least one of data type, current access frequency and importance index; A compression strategy determination module, used to determine a target compression strategy according to the memory usage information and the data attributes; A compression module is used to compress the data block according to the target compression strategy.

12. The memory compression device according to claim 11, characterized in that: The compression strategy determination module is specifically used to: Determine the current memory pressure of the data block according to the current memory usage information; Based on the corresponding relationship between the attribute and the limiting parameter, determining the compression rate limiting parameter according to the data attribute; Determine memory pressure limit parameters based on memory usage information in recent historical periods; Calculate the target compression rate according to the compression rate limit parameter, the memory pressure limit parameter and the current memory pressure; wherein the greater the current memory pressure, the greater the target compression rate; A target compression algorithm and compression parameters thereof are determined according to the target compression rate and the data type of the data block.

13. A memory compression device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the memory compression method according to any one of claims 1 to 10 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the memory compression method according to any one of claims 1 to 10.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the memory compression method according to any one of claims 1 to 10 is implemented.

Images