Data compression processing method and device, electronic equipment and medium

Abstract

This application discloses a data compression processing method, apparatus, electronic device, and medium, belonging to the field of data processing technology. The method includes: acquiring memory data of a page to be recycled; identifying the number of target fields in the memory data, where a target field is a field with n consecutive bytes of 0; determining that the page to be recycled belongs to a first type of compressed page if the number of target fields exceeds a first preset threshold; and performing memory recycling processing on the page to be recycled, wherein if the page to be recycled belongs to the first type of compressed page, the memory recycling processing includes compressing the memory data of the page to be recycled.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a data compression processing method, apparatus, electronic device, and medium. Background Technology

[0002] As applications are continuously developed, the number of applications installed on electronic devices is gradually increasing, and the memory usage of these applications is also growing. Some games and large artificial intelligence (AI) models often require more than 4GB of memory. When memory is insufficient, electronic devices compress memory data to Zipped RAM (zRAM) for reclamation. Current technology typically uses LZ4 to compress memory data first, and then determines whether it is a ZHP (Zero Page High) compressed large page or a ZMP (Zero Page Medium) compressed medium page based on the length of the compressed data. This allows the compression of ZHP and ZMP pages to be skipped during the reclamation process, reducing memory usage in high-load scenarios. However, the power consumption of this process is almost equivalent to a complete reclamation operation, and the compressed data is not saved but directly discarded, resulting in significant overhead. Summary of the Invention

[0003] The purpose of this application is to provide a data compression processing method, apparatus, electronic device, and medium that can solve the problem of high overhead in existing memory reclamation technologies.

[0004] In a first aspect, embodiments of this application provide a data compression processing method, the method comprising: Get the memory data of the pages to be reclaimed; Identify the number of target fields in the memory data, where the target field is a field with n consecutive bytes of 0, and n is an integer greater than 1; If the number of target fields is greater than a first preset threshold, the page to be recycled is determined to belong to the first type of compressed page; The memory reclamation process is performed on the page to be reclaimed, wherein, if the page to be reclaimed belongs to the first type of compressed page, the memory reclamation process includes compressing the memory data of the page to be reclaimed.

[0005] Secondly, embodiments of this application provide a data compression processing apparatus, including: The acquisition module is used to acquire the memory data of the pages to be reclaimed; The identification module is used to identify the number of target fields in the memory data, wherein the target field is a field with n consecutive bytes of 0, where n is an integer greater than 1; The first determining module is used to determine that the page to be recycled belongs to the first type of compressed page when the number of target fields is greater than a first preset threshold. The processing module is used to perform memory reclamation processing on the page to be reclaimed, wherein, when the page to be reclaimed belongs to the first type of compressed page, the memory reclamation processing includes compressing the memory data of the page to be reclaimed.

[0006] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0007] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0008] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0009] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0010] In this embodiment, the memory data of the page to be reclaimed is obtained; the number of target fields in the memory data is identified, where the target field is a field with n consecutive bytes of 0, and n is an integer greater than 1; if the number of target fields is greater than a first preset threshold, the page to be reclaimed is determined to belong to a first type of compressed page; memory reclamation processing is performed on the page to be reclaimed, wherein, if the page to be reclaimed belongs to the first type of compressed page, the memory reclamation processing includes compressing the memory data of the page to be reclaimed. Thus, by pre-determining the number of consecutive 0 bytes in the memory data of the page to be reclaimed, it is possible to quickly determine whether the compression process before memory reclamation can be skipped. Specifically, for first-type compressed pages with a large number of consecutive 0 bytes, there is no need for subsequent compression and post-compression length determination; instead, the page memory data is directly compressed during memory reclamation, thus eliminating the compression process before memory reclamation and reducing overhead. Therefore, compared with the prior art, this embodiment reduces overhead by directly performing a complete compression operation on the page to determine the page type before memory reclamation. Attached Figure Description

[0011] Figure 1 This is a flowchart of the data compression processing method provided in the embodiments of this application; Figure 2 This is a flowchart of the preprocessing process before memory reclamation provided in an embodiment of this application; Figure 3 This is a simplified compression flowchart provided in an embodiment of this application; Figure 4 This is a schematic diagram of the page recognition processing flow provided in the embodiments of this application; Figure 5 This is a structural diagram of the data compression processing apparatus provided in the embodiments of this application; Figure 6 This is a structural diagram of the electronic device provided in the embodiments of this application; Figure 7 This is a hardware structure diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] To make the embodiments of this application clearer, the relevant technical terms involved in the embodiments of this application are defined as follows: ZHP: Compressed large pages, pages with poor compression ratio, usually referring to pages that are larger than 75% of the original data size after compression. For such pages, the usual practice is not to compress them. ZMP: Compressed page, a page with a moderate compression ratio, usually refers to a page that is larger than 50% and less than 75% of the original data size after compression. This type of page can be compressed or not. ZSP: Compressed Small Page, a page with a good compression ratio, usually referring to a page that is less than 50% of the original data size after compression. It is recommended to compress such pages. LZ4: A lossless compression algorithm that offers fast compression and decompression speeds.

[0015] LZ4 main process: Hash: A 4-byte string is multiplied, shifted, and ANDed to obtain a fixed-length value, which is used to determine whether two strings are the same during matching.

[0016] Match: Find duplicate strings.

[0017] Update size (size): Used to record the size of the compressed data. It is often used to determine whether the expected output length is exceeded and is used as the final output of the function.

[0018] Encoding: The matched repeating string is replaced with a number in a specific field, which records the position and length of the preceding repeating string.

[0019] The data compression processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0020] Please see Figure 1 , Figure 1 A flowchart of the data compression processing method provided in the embodiments of this application is shown below. Figure 1 As shown, the method includes the following steps: Step 101: Obtain the memory data of the pages to be reclaimed.

[0021] The aforementioned pages to be reclaimed can refer to application pages running in the foreground or background of electronic devices. When a memory shortage alarm is triggered or the user performs a memory cleanup operation, these application pages need to be reclaimed, which means compressing the memory data of the pages to be reclaimed into ZRAM for recycling, in order to release the currently occupied memory.

[0022] In this embodiment of the application, before performing memory reclamation, the memory data of the page to be reclaimed can be obtained first, specifically the memory source data, in order to determine the type of the page to be reclaimed and to decide whether to compress the page memory source data during memory reclamation.

[0023] It should be noted that this application embodiment mainly uses memory data as the memory source data to illustrate the implementation scheme of this application. Here, memory source data refers to the original data temporarily residing in memory. For example, a segment of memory source data is shown in Table 1 below: Table 1 Memory Source Data

[0024] Step 102: Identify the number of target fields in the memory data. The target field is a field with n consecutive bytes of 0, where n is an integer greater than 1.

[0025] As can be seen from Table 1 above, some pages contain a large number of consecutive 0 values ​​in their memory source data. For compression algorithms, four or more consecutive bytes of "0" can be compressed. When there are a large number of consecutive 0 values, it can be basically determined that the page is not a ZHP page or a ZMP page. It can be compressed during memory reclamation, so there is no need to perform a compression process to estimate the compression length.

[0026] Therefore, in this embodiment, the load of the active identification stage can be reduced by adding a preprocessing process. This preprocessing process involves identifying the number of target fields in the memory source data of the page to be reclaimed. Based on whether the number of target fields is greater than a first preset threshold, it is determined whether the page to be reclaimed is a ZHP page or a ZMP page. The target field is a field where n or more consecutive bytes are 0. For example, in Table 1 above, if bytes 5 to 11 are all 0, it can be counted as one consecutive 0-byte field, and so on, until the total number of consecutive 0 bytes is counted.

[0027] Step 103: If the number of target fields is greater than the first preset threshold, determine that the page to be recycled belongs to the first type of compressed page.

[0028] The first preset threshold can be set based on experience. For example, the number of consecutive 0 bytes in a ZHP page and a ZMP page usually does not exceed 256, so 256 is used as the first preset threshold to determine whether a page is a ZHP page or a ZMP page. In this embodiment, pages can be divided into a first type of compressed page and a second type of compressed page based on the number of consecutive 0 values. The first type of compressed page is a page with a large number of consecutive 0 values ​​in the memory source data, which is considered to have a good compression ratio. It is recommended to compress such pages directly during memory reclamation. The second type of compressed page is a page with a small number of consecutive 0 values ​​in the memory source data, and its compression ratio is not yet determined. Further page identification and judgment are required.

[0029] In this step, the type of the page to be reclaimed can be determined by whether the number of target fields in the memory source data of the page to be reclaimed is greater than the first preset threshold. If the number of target fields is greater than the first preset threshold, the page to be reclaimed is determined to belong to the first type of compressed page, that is, a non-ZHP page and a non-ZMP page. In this case, the page memory source data can be compressed directly during memory reclamation.

[0030] Optionally, n equals 4, and the first preset threshold is greater than or equal to 256.

[0031] In some embodiments, when identifying the number of target fields in the memory source data of the page to be reclaimed, the memory source data can be scanned at least 4 bytes each time starting from the starting position to count the number of fields with 4 or more consecutive bytes of 0. In order to accurately distinguish between ZHP pages and ZMP pages, it can be determined whether the counted number of consecutive 0 bytes is greater than 256.

[0032] For example, the preprocessing procedure in the embodiments of this application can be as follows: Figure 2 As shown, the page memory source data is first scanned by bytes to count the number of consecutive 4 bytes of 0, m. If m is greater than 256, the current active identification process is exited to proceed to the next memory reclamation process. If m is not greater than 256, further page identification processing is performed.

[0033] Step 104: Perform memory reclamation processing on the page to be reclaimed, wherein, if the page to be reclaimed belongs to the first type of compressed page, the memory reclamation processing includes compressing the memory data of the page to be reclaimed.

[0034] In this step, memory reclamation can be performed on the page to be reclaimed based on the identified page type. Specifically, if it is determined that the page to be reclaimed belongs to the first type of compressed page, the memory source data of the page to be reclaimed can be compressed directly to perform memory reclamation without having to perform compression to determine the page attributes before memory reclamation.

[0035] If it is determined that the page to be recycled belongs to the second type of compressed page, the memory source data of the page to be recycled may not be compressed, or memory recycling may be performed based on the page attributes of the page to be recycled that are further identified.

[0036] Optionally, after step 102, the method further includes: If the number of target fields is less than or equal to the first preset threshold, the page to be recycled is determined to belong to the second type of compressed page, and page identification processing is performed on the page to be recycled to obtain the page attributes of the page to be recycled. Step 104 includes: If the page to be recycled belongs to the second type of compressed page, the page to be recycled is subjected to memory reclamation processing according to the identified page attributes of the page to be recycled.

[0037] In some embodiments, if the number of target fields is less than or equal to the first preset threshold, it can be determined that the page to be recycled belongs to the second type of compressed page. In this case, the page to be recycled may be a ZHP page or a ZMP page. At this time, further page identification processing can be performed on the page to be recycled to determine the specific page attributes.

[0038] Specifically, the page identification process can be to perform a compression operation on the page to be recycled to obtain the compressed length, and then determine whether the page is a ZHP page or a ZMP page based on the compressed length. Alternatively, the page to be recycled can be subjected to a simplified compression process, such as counting the fields with repeated byte content in the page's memory source data, and then estimating the compressed length of the page based on the statistical results. This can be achieved through hash calculation and matching without encoding operations, thereby reducing system overhead. Finally, the page is determined to be a ZHP page or a ZMP page based on the estimated compressed length.

[0039] During memory reclamation, the specific method for reclaiming the page can be determined based on the page identification result, i.e., the identified page attributes. For example, if the page to be reclaimed is identified as a ZHP page or a ZMP page, the memory source data of the page to be reclaimed is not compressed. Alternatively, if the page to be reclaimed is identified as a ZMP page, the memory source data of the page to be reclaimed is compressed. If the page to be reclaimed is neither a ZMP page nor a ZMP page, the memory source data of the page to be reclaimed is compressed.

[0040] In this way, for pages with a small number of consecutive 0 values, further page identification can be performed. Based on the page identification results, the memory reclamation processing method for the page can be determined. When the page memory data is large, compression processing is skipped. When the page memory data is small or moderate, compression processing is performed. This can reduce some unnecessary compression processing and help reduce memory usage in high-load scenarios.

[0041] Optionally, the page identification processing for the page to be recycled includes: Perform a hash operation on the memory data of the page to be reclaimed to determine the hash value corresponding to each byte index in the memory data; The hash value and byte content corresponding to each byte index in the memory data of the page to be reclaimed are matched, and the length of the compressed page to be reclaimed is estimated based on the matching results. If the length of the compressed page to be recycled is greater than or equal to a second preset threshold, the page to be recycled is marked as a page that can be skipped from compression.

[0042] In some embodiments, page identification processing of the page to be recycled can be performed by a simplified compression process, which omits the encoding step compared to the existing compression process. That is, only the memory source data of the page to be recycled needs to be hashed and matched to estimate the length of the compressed page, without the need for subsequent encoding compression.

[0043] Currently, the active identification strategy requires completing the entire compression process. However, only the size of the compressed data is needed to determine whether a page is a ZHP or ZMP page, and the compressed data does not need to be decompressed subsequently. Therefore, the encoding process can be removed, and the simplified LZ compression process can be modified as follows: Figure 3 As shown.

[0044] Specifically, a hash operation can be performed on the memory source data of the page to be reclaimed, calculating the hash value of each byte position. For example, multiplication, shifting, and other calculations can be performed on the first four bytes starting from the current position. Each byte is recorded according to its position index, such as the index of the first byte being 1 and the index of the second byte being 2. Then, the calculated byte indexes and corresponding hash values ​​are recorded in a hash table, and a correspondence is established. Taking the string "abcdefgzbcdefgy" as an example, the hash values ​​of index 1 and index 2 are calculated as h1(abcd) and h2(bcde), respectively, and the indices and corresponding hash values ​​are recorded in the hash table, establishing the following correspondence: index 1 corresponds to h1(abcd), and index 2 corresponds to h2(bcde).

[0045] Then, the hash values ​​of each byte index are matched, and the contents of bytes with the same hash value are further compared. If they are equal, the match is successful; if they are not equal, the match fails. The matching continues until the longest matching byte length is found. The contents of the successfully matched bytes are equal and can be compressed. Therefore, the length of the compressed page can be estimated based on the matching results.

[0046] Taking the string "abcdefgzbcdefgy" as an example, the first four bytes are compared with the four bytes corresponding to the same hash value in the hash table. If they are equal, the match is successful. Suppose h1(abcd) = h6(fgzb), but the comparison shows that abcd is not equal to fgzb, so the match fails. We can see that the hash value at index 2 is equal to the hash value at index 9, and the four bytes are also equal, i.e., h2(bcde) = h9(bcde), so the match is successful. To obtain the longest possible match length, the system continues to match the bytes of both strings. If the bytes are equal, the matching continues. For example, if the bytes after bcde in both instances are also equal, then the current longest possible match length is 6.

[0047] Based on the matching results, the compressed length of the page to be reclaimed can be estimated. A longer longest matching length results in a larger compressible byte length, thus a smaller compressed length, and vice versa. Based on the estimated compressed length, a comparison with a second preset threshold can be made to determine whether the page to be reclaimed is a skippable compression page, i.e., a page type that can skip compression during memory reclamation. Specifically, if the estimated compressed length is greater than or equal to the second preset threshold, the page to be reclaimed can be marked as a skippable compression page, i.e., either a ZHP page or a ZMP page. If the estimated compressed length is less than the second preset threshold, it indicates that the page to be reclaimed is neither a ZHP page nor a ZMP page, but a compressed small page, which can be compressed during memory reclamation.

[0048] The second preset threshold can be set based on experience. For example, the length of ZHP and ZMP pages after compression usually exceeds 2048, so 2048 can be used as the second preset threshold to determine whether the page is a ZHP page or a ZMP page based on whether the length of the page after compression is greater than 2048.

[0049] Optionally, the matching result includes the offset between the successfully matched byte indices, the longest matching length, and the length of the string that cannot be compressed.

[0050] In some embodiments, when matching the hash values ​​and byte contents corresponding to each byte index in the memory source data of the page to be reclaimed, the longest matching length can be obtained based on the successfully matched bytes, and the offset between the successfully matched byte indices can be determined based on the difference between them. Furthermore, the length of the uncompressible string can be determined based on the unmatched bytes. For example, in the string "abcdefgzbcdefgy", the 4 bytes "bcde" corresponding to index 2 match the 4 bytes "bcde" corresponding to index 9, and the following 2 bytes "fg" also match. Therefore, the longest matching length is 4 + 2 = 6, the offset between the two indices is 9 - 2 = 7, and the length of the uncompressible string "abcdefgz" is 8. Based on this information, the length of the data that needs to be stored can be calculated, which is the length of the string after compression, without actually performing encoding compression.

[0051] Further, the step of marking the page to be recycled as a page that can be skipped in compression when the length of the compressed page to be recycled is greater than or equal to a second preset threshold includes: If the length of the compressed page to be recycled is greater than or equal to the second preset threshold and less than the third preset threshold, the page to be recycled is marked as a compressed ZMP page. If the length of the compressed page to be recycled is greater than the third preset threshold, the page to be recycled is marked as a compressed large page (ZHP page), where the third preset threshold is greater than the second preset threshold.

[0052] In some embodiments, two different thresholds can be used to further distinguish between ZHP pages and ZMP pages. Since ZHP pages have a lower compression ratio and a longer compressed length than ZMP pages, a larger third preset threshold can be introduced to identify ZHP pages. For example, the second preset threshold is 2048, and the third preset threshold is 3072. Thus, if the estimated compressed length of the page to be recycled is greater than or equal to the second preset threshold but less than the third preset threshold, the page to be recycled can be marked as a ZMP page. If the estimated compressed length of the page to be recycled is greater than both the second and third preset thresholds, the page to be recycled can be marked as a ZHP page. In this way, by combining two different length thresholds, ZHP pages and ZMP pages can be identified more accurately.

[0053] Combining the above preprocessing and simplified compression processes, the optimized page active recognition process of this application embodiment can be obtained as follows: Figure 4 As shown.

[0054] Optionally, when the page to be recycled belongs to the second type of compressed page, performing memory reclamation processing on the page to be recycled according to the identified page attributes of the page to be recycled includes: If the page to be reclaimed is a ZHP page or a ZMP page, skip the compression process of the memory data of the page to be reclaimed. If the length of the compressed page to be recycled is less than the second preset threshold, the memory data of the page to be recycled is compressed.

[0055] In some embodiments, for pages such as ZHP and ZMP, which have poor compression ratios, compression processing can be skipped during memory reclamation to reduce CPU usage in high-load scenarios.

[0056] For pages whose compressed length is less than the second preset threshold, which are usually compressed small pages such as ZSP pages, the compression ratio of these pages is better. Therefore, the memory source data of the pages to be reclaimed can be compressed during memory reclamation to free up more memory.

[0057] This application optimizes the active identification process before page reclamation, which can skip about 65% of page compression, thereby reducing the load of the active identification process by about 40%, and the identification accuracy reaches more than 93%, without affecting subsequent memory reclamation.

[0058] The data compression processing method in this embodiment of the application obtains the memory data of the page to be reclaimed; identifies the number of target fields in the memory data, where the target field is a field with n consecutive bytes of 0, and n is an integer greater than 1; if the number of target fields is greater than a first preset threshold, it is determined that the page to be reclaimed belongs to a first type of compressed page; and performs memory reclamation processing on the page to be reclaimed. Specifically, if the page to be reclaimed belongs to the first type of compressed page, the memory reclamation processing includes compressing the memory data of the page to be reclaimed. In this way, by pre-determining the number of consecutive 0 bytes in the memory data of the page to be reclaimed, it is possible to quickly determine whether the compression process before memory reclamation can be skipped. Specifically, for first-type compressed pages with a large number of consecutive 0 bytes, there is no need for subsequent compression and post-compression length determination; instead, the page memory data is directly compressed during memory reclamation, thus eliminating the compression process before memory reclamation and reducing overhead. Therefore, compared with the prior art, this embodiment of the application reduces overhead by directly performing a complete compression operation on the page to determine the page type before performing memory reclamation.

[0059] The data compression processing method provided in this application can be executed by a data compression processing device. This application uses an example of a data compression processing device executing the data compression processing method to illustrate the data compression processing device provided in this application.

[0060] Please see Figure 5 , Figure 5 This is a schematic diagram of the data compression processing apparatus provided in the embodiments of this application, as shown below. Figure 5 As shown, the data compression processing apparatus 500 includes: Module 501 is used to obtain the memory data of the page to be reclaimed; The identification module 502 is used to identify the number of target fields in the memory data, wherein the target field is a field with n consecutive bytes of 0, where n is an integer greater than 1; The first determining module 503 is used to determine that the page to be recycled belongs to the first type of compressed page when the number of target fields is greater than a first preset threshold. The processing module 504 is used to perform memory reclamation processing on the page to be reclaimed, wherein, when the page to be reclaimed belongs to the first type of compressed page, the memory reclamation processing includes compressing the memory data of the page to be reclaimed.

[0061] Optionally, the data compression processing apparatus 500 further includes: The second determining module is used to determine that the page to be recycled belongs to the second type of compressed page when the number of the target fields is less than or equal to the first preset threshold, and to perform page identification processing on the page to be recycled to obtain the page attributes of the page to be recycled when the page to be recycled belongs to the second type of compressed page. The processing module 504 is further configured to perform memory reclamation processing on the page to be reclaimed according to the identified page attributes of the page to be reclaimed when the page to be reclaimed belongs to the second type of compressed page.

[0062] Optionally, the second determining module includes: The determining unit is used to perform a hash operation on the memory data of the page to be reclaimed, and to determine the hash value corresponding to each byte index in the memory data; The estimation unit is used to match the hash value and byte content corresponding to each byte index in the memory data of the page to be reclaimed, and to estimate the length of the compressed page to be reclaimed based on the matching result. The marking unit is used to mark the page to be recycled as a page that can be skipped in compression if the length of the compressed page to be recycled is greater than or equal to a second preset threshold.

[0063] Optionally, the marking unit is used for: If the length of the compressed page to be recycled is greater than or equal to the second preset threshold and less than the third preset threshold, the page to be recycled is marked as a compressed ZMP page. If the length of the compressed page to be recycled is greater than the third preset threshold, the page to be recycled is marked as a compressed large page (ZHP page), where the third preset threshold is greater than the second preset threshold.

[0064] Optionally, the matching result includes the offset between the successfully matched byte indices, the longest matching length, and the length of the string that cannot be compressed.

[0065] Optionally, the processing module 504 includes: The first processing unit is configured to skip the compression processing of the memory data of the page to be reclaimed when the page to be reclaimed is a ZHP page or a ZMP page. The second processing unit is used to compress the memory data of the page to be recycled if the length of the compressed page is less than the second preset threshold.

[0066] Optionally, n equals 4, and the first preset threshold is greater than or equal to 256.

[0067] The data compression processing apparatus 500 in this embodiment acquires the memory data of a page to be reclaimed; identifies the number of target fields in the memory data, where the target field is a field with n consecutive bytes of 0, and n is an integer greater than 1; if the number of target fields is greater than a first preset threshold, it determines that the page to be reclaimed belongs to a first type of compressed page; and performs memory reclamation processing on the page to be reclaimed. Specifically, if the page to be reclaimed belongs to the first type of compressed page, the memory reclamation processing includes compressing the memory data of the page to be reclaimed. Thus, by pre-determining the number of consecutive 0 bytes in the memory data of the page to be reclaimed, it is possible to quickly determine whether the compression process before memory reclamation can be skipped. Specifically, for first-type compressed pages with a large number of consecutive 0 bytes, there is no need for subsequent compression and length determination after compression; instead, the page memory data is directly compressed during memory reclamation, thus eliminating the compression process before memory reclamation and reducing overhead. Therefore, compared to the prior art method of directly performing a complete compression operation on the page to determine the page type before memory reclamation, this embodiment reduces overhead.

[0068] The data compression processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0069] The data compression processing device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0070] The data compression processing device provided in this application embodiment can achieve... Figures 1 to 4 The various processes implemented in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0071] Optionally, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores a program or instructions that can run on the processor 601. When the program or instructions are executed by the processor 601, they implement the various steps of the above-described data compression processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0072] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0073] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0074] The electronic device 700 includes, but is not limited to, components such as: radio frequency unit 701, network module 702, audio output unit 703, input unit 704, sensor 705, display unit 706, user input unit 707, interface unit 708, memory 709, and processor 710.

[0075] Those skilled in the art will understand that the electronic device 700 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 710 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0076] The processor 710 is used for: Get the memory data of the pages to be reclaimed; Identify the number of target fields in the memory data, where the target field is a field with n consecutive bytes of 0, and n is an integer greater than 1; If the number of target fields is greater than a first preset threshold, the page to be recycled is determined to belong to the first type of compressed page; The memory reclamation process is performed on the page to be reclaimed, wherein, if the page to be reclaimed belongs to the first type of compressed page, the memory reclamation process includes compressing the memory data of the page to be reclaimed.

[0077] Optionally, the processor 710 is also used for: If the number of target fields is less than or equal to the first preset threshold, it is determined that the page to be recycled belongs to the second type of compressed page. If the page to be recycled belongs to the second type of compressed page, page identification processing is performed on the page to be recycled to obtain the page attributes of the page to be recycled. If the page to be recycled belongs to the second type of compressed page, the page to be recycled is subjected to memory reclamation processing according to the identified page attributes of the page to be recycled.

[0078] Optionally, the processor 710 is also used for: Perform a hash operation on the memory data of the page to be reclaimed to determine the hash value corresponding to each byte index in the memory data; The hash value and byte content corresponding to each byte index in the memory data of the page to be reclaimed are matched, and the length of the compressed page to be reclaimed is estimated based on the matching results. If the length of the compressed page to be recycled is greater than or equal to a second preset threshold, the page to be recycled is marked as a page that can be skipped from compression.

[0079] Optionally, the processor 710 is also used for: If the length of the compressed page to be recycled is greater than or equal to the second preset threshold and less than the third preset threshold, the page to be recycled is marked as a compressed ZMP page. If the length of the compressed page to be recycled is greater than the third preset threshold, the page to be recycled is marked as a compressed large page (ZHP page), where the third preset threshold is greater than the second preset threshold.

[0080] Optionally, the matching result includes the offset between the successfully matched byte indices, the longest matching length, and the length of the string that cannot be compressed.

[0081] Optionally, the processor 710 is also used for: If the page to be reclaimed is a ZHP page or a ZMP page, skip the compression process of the memory data of the page to be reclaimed. If the length of the compressed page to be recycled is less than the second preset threshold, the memory data of the page to be recycled is compressed.

[0082] Optionally, n equals 4, and the first preset threshold is greater than or equal to 256.

[0083] It should be understood that, in this embodiment, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042. The GPU 7041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0084] The memory 709 can be used to store software programs and various data. The memory 709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0085] Processor 710 may include one or more processing units; optionally, processor 710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 710.

[0086] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described data compression processing method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0087] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0088] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described data compression processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0089] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0090] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the data compression processing method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0091] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0093] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A data compression processing method, characterized in that, include: Get the memory data of the pages to be reclaimed; Identify the number of target fields in the memory data, where the target field is a field with n consecutive bytes of 0, and n is an integer greater than 1; If the number of target fields is greater than a first preset threshold, the page to be recycled is determined to belong to the first type of compressed page; The memory reclamation process is performed on the page to be reclaimed, wherein, if the page to be reclaimed belongs to the first type of compressed page, the memory reclamation process includes compressing the memory data of the page to be reclaimed.

2. The method according to claim 1, characterized in that, The method further includes: If the number of target fields is less than or equal to the first preset threshold, the page to be recycled is determined to belong to the second type of compressed page, and page identification processing is performed on the page to be recycled to obtain the page attributes of the page to be recycled. The memory reclamation process for the pages to be reclaimed includes: If the page to be recycled belongs to the second type of compressed page, the page to be recycled is subjected to memory reclamation processing according to the identified page attributes of the page to be recycled.

3. The method according to claim 2, characterized in that, The page identification process for the page to be recycled includes: Perform a hash operation on the memory data of the page to be reclaimed to determine the hash value corresponding to each byte index in the memory data; The hash value and byte content corresponding to each byte index in the memory data of the page to be reclaimed are matched, and the length of the compressed page to be reclaimed is estimated based on the matching results. If the length of the compressed page to be recycled is greater than or equal to a second preset threshold, the page to be recycled is marked as a page that can be skipped from compression.

4. The method according to claim 3, characterized in that, The step of marking the page to be recycled as a page that can be skipped from compression when the length of the compressed page is greater than or equal to a second preset threshold includes: If the length of the compressed page to be recycled is greater than or equal to the second preset threshold and less than the third preset threshold, the page to be recycled is marked as a compressed ZMP page. If the length of the compressed page to be recycled is greater than the third preset threshold, the page to be recycled is marked as a compressed large page (ZHP page), where the third preset threshold is greater than the second preset threshold.

5. The method according to claim 4, characterized in that, When the page to be recycled belongs to the second type of compressed page, the memory reclamation process for the page to be recycled is performed according to the identified page attributes of the page to be recycled, including: If the page to be reclaimed is a ZHP page or a ZMP page, skip the compression process of the memory data of the page to be reclaimed. If the length of the compressed page to be recycled is less than the second preset threshold, the memory data of the page to be recycled is compressed.

6. A data compression processing apparatus, characterized in that, include: The acquisition module is used to acquire the memory data of the pages to be reclaimed; The identification module is used to identify the number of target fields in the memory data, wherein the target field is a field with n consecutive bytes of 0, where n is an integer greater than 1; The first determining module is used to determine that the page to be recycled belongs to the first type of compressed page when the number of target fields is greater than a first preset threshold. The processing module is used to perform memory reclamation processing on the page to be reclaimed, wherein, when the page to be reclaimed belongs to the first type of compressed page, the memory reclamation processing includes compressing the memory data of the page to be reclaimed.

7. The data compression processing apparatus according to claim 6, characterized in that, The data compression processing device further includes: The second determining module is used to determine that the page to be recycled belongs to the second type of compressed page when the number of the target fields is less than or equal to the first preset threshold, and to perform page identification processing on the page to be recycled. The processing module is used to perform memory reclamation processing on the page to be reclaimed according to the identified page attributes of the page to be reclaimed when the page to be reclaimed belongs to the second type of compressed page.

8. The data compression processing apparatus according to claim 7, characterized in that, The second determining module includes: The determining unit is used to perform a hash operation on the memory data of the page to be reclaimed, and to determine the hash value corresponding to each byte index in the memory data; The estimation unit is used to match the hash value and byte content corresponding to each byte index in the memory data of the page to be reclaimed, and to estimate the length of the compressed page to be reclaimed based on the matching result. The marking unit is used to mark the page to be recycled as a page that can be skipped in compression if the length of the compressed page to be recycled is greater than or equal to a second preset threshold.

9. The data compression processing apparatus according to claim 8, characterized in that, The marking unit is used for: If the length of the compressed page to be recycled is greater than or equal to the second preset threshold and less than the third preset threshold, the page to be recycled is marked as a compressed ZMP page. If the length of the compressed page to be recycled is greater than the third preset threshold, the page to be recycled is marked as a compressed large page (ZHP page), where the third preset threshold is greater than the second preset threshold.

10. The data compression processing apparatus according to claim 9, characterized in that, The processing module includes: The first processing unit is configured to skip the compression processing of the memory data of the page to be reclaimed when the page to be reclaimed is a ZHP page or a ZMP page. The second processing unit is used to compress the memory data of the page to be recycled if the length of the compressed page is less than the second preset threshold.

11. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the data compression processing method as described in any one of claims 1 to 5.

12. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the data compression processing method as described in any one of claims 1 to 5.

Images