Data processing method, device, equipment, readable storage medium and program product

Abstract

The present application provides a data processing method, device, equipment, readable storage medium and program product. The method includes: receiving the length information of the compressed data sent by the hardware compression device, and obtaining the remaining physical space of the memory according to the length information; obtaining the free space in the logical space of the memory; adjusting the free space according to the remaining physical space; and writing data to the memory according to the adjusted free space. The method of the present application can not only avoid write errors caused by excessive logical space and insufficient physical space in the memory with hardware compression through dynamic adjustment of the logical space, but also improve the space utilization of the memory with hardware compression.

Description

Technical Field

[0001] The present application relates to computer technology, and in particular to a data processing method, device, equipment, readable storage medium and program product. Background Art

[0002] During the use of the computer, the size of the memory has a great impact on the computer's performance. A larger memory can bring a higher operating speed. In addition to purchasing new memory to increase memory space, you can also increase the amount of free memory through data compression.

[0003] At present, data can be compressed by the central processing unit (CPU), and this software compression method releases memory space. However, software compression will not only take up additional CPU resources, but also reduce the operating efficiency of the CPU. In order to solve this problem, a hardware compression method can be used. Hardware compression is to use a special hardware compression device to compress the data in the cache before writing the data from the cache to the memory, and write the compressed data to the memory according to the remaining logical capacity, wherein the hardware compression device includes a device with a built-in hardware compression chip. This hardware compression method does not occupy CPU resources and can improve the operating efficiency of the CPU. In the above-mentioned hardware compression method, the logical capacity of the memory is pre-configured before use. For the upper-level software of the electronic device, during the data writing process, the electronic device will determine the current remaining logical capacity based on the pre-configured logical capacity, and write data according to the current remaining logical capacity.

[0004] However, the above data writing process may cause data writing errors or low memory utilization. Summary of the invention

[0005] The present application provides a data processing method, apparatus, device, readable storage medium and program product, which are used to solve the problem of data writing errors and low memory utilization caused by the inability to change logical capacity.

[0006] In a first aspect, the present application provides a data processing method, comprising:

[0007] Receive the length information of the compressed data sent by the hardware compression device, and obtain the remaining physical space of the memory according to the length information; obtain the free space in the logical space of the memory; adjust the free space according to the remaining physical space; write data into the memory according to the adjusted free space.

[0008] In a possible implementation, adjusting the free space according to the remaining physical space includes:

[0009] A first difference between the remaining physical space and the free space is determined, and the free space is adjusted according to the first difference.

[0010] In a possible implementation, adjusting the free space according to the first difference includes:

[0011] Determine whether the remaining physical space is greater than the free space; if the remaining physical space is greater than the free space, expand the free space according to the first difference; if the remaining physical space is less than the free space, reduce the free space according to the first difference.

[0012] In a possible implementation, the data processing method further includes:

[0013] The physical address at the end of the last writing of data into the memory is obtained, and the address mapping table is updated according to the physical address and length information, wherein the address mapping table includes the correspondence between the logical address and the physical address of the data stored in the memory, the logical address is the address in the logical space, and the physical address is the address in the physical space. Wherein, writing data into the memory according to the adjusted free space includes: writing data into the memory according to the free space and the updated address mapping table.

[0014] In a possible implementation, the data processing method further includes:

[0015] When part of the logical space in the logical space is in a released state, the free space is updated according to the part of the logical space, the physical space corresponding to the part of the logical space is determined according to the address mapping table, and the remaining physical space is updated according to the physical space corresponding to the part of the logical space; wherein, adjusting the free space according to the remaining physical space includes: adjusting the updated free space according to the updated remaining physical space.

[0016] In a possible implementation, the data processing method further includes:

[0017] Obtain the target logical address of the data to be processed; determine the target physical address corresponding to the target logical address according to the updated address mapping table, read the data to be processed from the target physical address, decompress the data to be processed, and send the decompressed data to the cache.

[0018] In a second aspect, the present application provides a data processing device, including:

[0019] A receiving module, used for receiving the length information of the compressed data sent by the hardware compression device;

[0020] An acquisition module is used to obtain the remaining physical space of the memory according to the length information;

[0021] The acquisition module is also used to obtain the free space in the logical space of the memory;

[0022] A processing module, used for adjusting the free space according to the remaining physical space;

[0023] The processing module is further used to write data into the memory according to the adjusted free space.

[0024] In a possible implementation, the processing module is specifically configured to:

[0025] A first difference between the remaining physical space and the free space is determined, and the free space is adjusted according to the first difference.

[0026] In a possible implementation, the processing module is specifically configured to:

[0027] Determine whether the remaining physical space is greater than the free space;

[0028] If the remaining physical space is greater than the free space, the free space is expanded according to the first difference;

[0029] If the remaining physical space is less than the free space, the free space is reduced according to the first difference.

[0030] In a possible implementation, the acquisition module is further used to: acquire the physical address when the previous writing of data to the memory ends.

[0031] The processing module is further used to: update the address mapping table according to the physical address and length information, the address mapping table including the correspondence between the logical address and the physical address of the data stored in the memory, the logical address is the address in the logical space, and the physical address is the address in the physical space;

[0032] Processing module, specifically used for:

[0033] Write data to the memory based on the free space and the updated address mapping table.

[0034] In a possible implementation, the processing module is further used to: when part of the logical space is in a released state, update the free space according to the part of the logical space;

[0035] Determine the physical space corresponding to the portion of the logical space according to the address mapping table;

[0036] Update the remaining physical space according to the physical space corresponding to the part of the logical space;

[0037] Adjust the updated free space according to the updated remaining physical space.

[0038] In a possible implementation, the acquisition module is further used to: acquire a target logical address of the data to be processed.

[0039] The processing module is also used to: obtain the target logical address of the data to be processed; determine the target physical address corresponding to the target logical address according to the updated address mapping table; read the data to be processed from the target physical address and decompress the data to be processed; and send the decompressed data to the cache.

[0040] In a third aspect, the present application provides a data processing device, including: a processor, a memory;

[0041] Memory is used to store computer executable instructions.

[0042] The processor is used to execute the computer execution instructions stored in the memory to implement the data processing method of the first aspect.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the data processing method of the first aspect.

[0044] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the data processing method of any possible implementation method of the above-mentioned first aspect.

[0045] The data processing method, apparatus, device, readable storage medium and program product provided by the present application receive the length information of compressed data sent by a hardware compression device, and obtain the remaining physical space of the memory according to the length information; obtain the free space in the logical space of the memory, and adjust the free space according to the remaining physical space, and after completing the adjustment of the free space, write data into the memory according to the adjusted free space, thereby achieving the effect of dynamic adjustment of the free space, avoiding data writing errors caused by too large logical space and insufficient actual physical space, and the problem of low memory utilization caused by too small logical space and free actual physical space, thereby achieving full utilization of the memory space. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the system architecture of the data processing method provided in this application;

[0047] Figure 2 A schematic diagram of a flow chart of a data processing method provided in an embodiment of the present application;

[0048] Figure 3 Another flowchart of a data processing method provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of an address mapping table provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a data processing device provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0053] The data processing method provided in the embodiment of the present application can be applied to the data storage process of personal computers, servers, workstations and clusters. Among them, the data storage can be the data storage of the system itself or the storage of application data.

[0054] At present, in order to save memory space, data compression is usually used to reduce memory usage. The compression method currently used mainly uses the central processing unit (CPU) for compression, such as zRAM, zSwap and zCache. These software-based compression solutions not only occupy additional CPU resources, but also involve a large amount of data movement. Since software compression occupies the CPU, the actual performance is poor. It can only be used in mobile devices or embedded devices, and cannot be applied to high-performance workstations / servers. Based on this, a hardware compression method is currently proposed. Specifically, a compression and decompression module is integrated in the memory controller. When data is written from the CPU internal cache (such as L3 cache) by the memory controller to the memory particle, the data is first compressed by the compression module, and then the compressed data is written to the memory particle. When the data is read from the memory particle by the memory controller, the data is first decompressed by the decompression module and then sent to the CPU internal cache. Among them, the logical capacity of the memory is pre-configured before use. For the upper-level software of the electronic device, it does not participate in the data compression process. Therefore, during the data writing process, the electronic device will determine the current remaining logical capacity based on the pre-configured logical capacity and the amount of data that needs to be written before compression, and write data according to the current remaining logical capacity. However, when the remaining logical capacity is greater than the remaining physical capacity, the physical capacity will be exhausted, but the logical capacity will still be remaining. At this time, when writing data to the memory according to the remaining logical capacity, data overflow will occur, and the memory will report an error, which will cause an application execution error. When the remaining logical capacity is less than the remaining physical capacity, the logical capacity will be exhausted, but the physical capacity will still be remaining. At this time, the operating system will think that the memory is full and will no longer write data to the memory, resulting in low memory utilization.

[0055] For example, if the physical capacity of the memory is 80G and the logical capacity is 200G, when the electronic device needs to write 100G of data into the memory through the operating system, assuming that the compression ratio of the data after hardware compression is 2:1, the data actually stored in the memory is 50G. At this time, the current physical remaining capacity of the memory is 30G, but the operating system of the electronic device does not participate in the above-mentioned hardware compression, and its current logical remaining capacity is determined to be 100G. Therefore, the electronic device may later write another 100G of data into the memory through the operating system. After compression by the compression module (assuming the compression ratio is still 2:1), the data is 50G, but the current remaining physical capacity is only 30G. This will cause data overflow in the memory space and cause data writing errors. If the physical capacity of the memory is 100G and the logical capacity is 100G, when the electronic device needs to write 50G of data into the memory through the operating system, after compressing the data through hardware compression (compression ratio of 2:1), the data actually stored in the memory is 25G. At this time, the current physical remaining capacity of the memory is 75G, but the operating system of the electronic device does not participate in the above-mentioned hardware compression, and the current logical remaining capacity determined by it is 50G. Therefore, the electronic device will only write 50G of data into the memory through the operating system. This will cause a waste of memory space and low memory utilization.

[0056] Taking the above-mentioned technical problems into consideration in the embodiments of the present application, a data processing method is proposed, in which the size of the logical space can be adjusted in real time according to the actual remaining physical space in the memory, and data can be written to the memory based on the adjusted logical space, thereby avoiding data writing errors caused by too large logical space and insufficient actual physical space and low memory utilization caused by too small logical space and free actual physical space, thereby achieving full utilization of the memory space.

[0057] Before describing the data processing method of this application, Figure 1 Learn about the sample operating environment for this application.

[0058] Figure 1 A schematic diagram of the system architecture of the data processing method provided in this application, such as Figure 1As shown, the system includes a processor 101, a traditional memory 102, and a memory 103 with hardware compression, wherein the memory 103 with hardware compression includes a memory controller 104 and a memory 105, and the memory controller 104 includes a compression module 106 and a decompression module 107. The traditional memory 102 and the memory 105 can both be used to store the address mapping table and the physical address at the end of data writing, wherein the address mapping table records the correspondence between the logical address and the physical address. When data needs to be stored in the memory, the processor 101 will read the address mapping table and the physical address at the end of the last write from the traditional memory 102 or the memory 105, and send the data to be stored and the physical address at the end of the last write to the memory 103 with hardware compression for storage, and when data needs to be read, the processor needs to read the address mapping table and convert the target logical address of the data to be processed into the target physical address, send the target physical address to the memory 103 with hardware compression, and receive the decompressed data returned by the memory 103 with hardware compression. The memory controller can store data by compressing the data through the compression module 106 and then writing it into the memory 105. The processor 101 can also receive decompressed data from the memory controller 104. The decompressed data can be obtained by decompressing the data in the memory using the decompression module 107.

[0059] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0060] Figure 2 This is a flow chart of a data processing method provided in an embodiment of the present application. The present application provides a data processing method, which can be executed by any device that executes the data processing method. The device can be implemented by software and / or hardware. For example, the device can be an electronic device or a chip integrated in an electronic device. Figure 2 As shown, the data processing method provided in the embodiment of the present application includes the following steps:

[0061] Step 201: Receive the length information of the compressed data sent by the hardware compression device, and obtain the remaining physical space of the memory according to the length information.

[0062] In this step, the hardware compression device may include a memory controller, an integrated circuit capable of implementing data compression, etc. After the hardware compression device compresses the data to be stored or written into the memory by hardware compression, the length information of the compressed data may be returned to the upper software of the electronic device, for example, to the operating system.

[0063] Since the physical address of the data written after each writing to the memory needs to be recorded, when the electronic device obtains the compressed data length and the physical address after the last writing, it can determine the physical space currently used by the memory, and then based on the total physical space of the memory, it can determine the remaining physical space of the memory.

[0064] Exemplarily, the physical space of the memory can be understood as the actual space in the memory.

[0065] Step 202: Obtain free space in the logical space of the memory.

[0066] In this step, the logical space corresponds to the physical space, which is the space fed back by the memory to the upper-level software. It can also be understood as the space exposed to the operating system, or the space considered by the operating system as the memory. Usually, when an electronic device writes data to the memory through the operating system, it is written based on the size of the logical space.

[0067] The free space in the logical space may include unoccupied space in the logical space, which may be obtained by subtracting the amount of data written this time from the free space remaining after the last data was written.

[0068] Step 203: According to the remaining physical space, the free space is adjusted so that the difference between the remaining physical space and the adjusted free space is smaller than a preset value.

[0069] In this step, after data is stored, the size of the remaining physical space may be larger than the free space, or may be smaller than the free space. Since the electronic device writes data to the memory through the operating system, it is written based on the logical space. Therefore, in order to better utilize the physical space, it is necessary to adjust the size of the free space in the logical space according to the remaining physical space. In order to prevent the situation where the amount of memory occupied by the written data is greater than the physical memory, or the situation where the logical space is exhausted but the physical space is still available, the difference between the remaining physical space and the adjusted free space is less than the preset value, wherein the preset value can be set according to actual conditions or experience, for example, it can be set to 0.5G or 0.3G. It should be understood that the smaller the preset value, the closer the remaining physical space is to the adjusted free space, so that the utilization rate of the physical space of the memory is higher.

[0070] In order to quickly adjust the free space and make the adjusted free space close to the remaining physical space so that the physical space can be fully utilized, in a possible implementation method, the free space is adjusted according to the remaining physical space. The first difference between the remaining physical space and the free space can be determined, and the free space is adjusted according to the first difference.

[0071] Specifically, the remaining physical space is the unoccupied physical space, and the free space is the unoccupied space in the logical space. The difference between the two is the first difference. According to the value of the first difference, it can be determined how large the free space can be adjusted. Exemplarily, when adjusting the free space according to the first difference, it can be determined whether the remaining physical space is larger than the free space. If the remaining physical space is larger than the free space, the free space is expanded according to the first difference. If the remaining physical space is smaller than the free space, the free space is reduced according to the first difference.

[0072] Specifically, the electronic device can determine the size of the remaining physical space and the free space. When the remaining physical space is larger than the free space, the free space size will be increased to make the free space equal to or close to the remaining physical space. For example, for a memory with hardware compression of 100GB in physical space, the size of the initialized logical space is also 100GB. At a data compression ratio of 2:1 (the size before compression is twice the size after compression), 60GB of data is written to the memory, and 30GB of physical space is actually occupied. At this time, the free space in the logical space is (100GB-60GB=40GB), and the remaining physical space is (100GB-30GB=70GB). The difference between the remaining physical space and the free space in the logical space is 30GB. Since the remaining physical space is larger than the free space, the free space can be expanded by 30GB, and the adjusted free space is 70GB. In this way, when writing data later, data will be written to the memory based on the adjusted free space of 70GB.

[0073] When the compressed data is updated, the remaining physical space may be less than the free space. In this case, the free space size will be reduced by the first difference value to make the free space equal to the remaining physical space. For example, when 40GB of the previously written 60GB data is updated and the updated data compression ratio is 1:1, the physical space occupied at this time is (20GB / 2+40GB=50GB), the remaining physical space is (100GB–50GB=50GB), and the free space is 70GB. The difference between the remaining physical space and the free space in the logical space is 20GB. Since the remaining physical space is less than the free space, the free space can be reduced by 20GB, and the adjusted free space is 50GB. In this way, when writing data later, data will be written to the memory based on the adjusted free space of 50GB.

[0074] In this embodiment, by comparing the size of the remaining physical space and the free space, the effect of increasing the free space size when the remaining physical space is larger than the free space and reducing the free space when the remaining physical space is smaller than the free space is achieved. Ultimately, the difference between the remaining physical space and the logical space is smaller than a preset value, ensuring that data can be correctly written into the memory while maximizing the use of the memory space.

[0075] In another possible implementation, when adjusting the free space according to the remaining physical space, the data compression rate when the hardware compression device compresses data may be further considered to adjust the free space according to the remaining physical space and the above data compression rate.

[0076] Specifically, when the operating system is initialized, the electronic device can use a logical space that is no smaller than the remaining physical space of the detected memory with hardware compression. The logical space can be divided into occupied space and free space, and the occupied space is further divided into actual occupied space and marked occupied space.

[0077] The electronic device first obtains the size and compression ratio of the current remaining physical space, and uses the size and compression ratio of the current remaining physical space to adjust the free space. For example, when the size of the remaining physical space is 30GB and the compression ratio is 2:1, the size of the free space can be adjusted to 60GB. The compression ratio can be the default value of the hardware compression device, or the ratio of the current actual occupied space size to the total physical space minus the remaining physical space, or the compression ratio value obtained in real time during the data compression process. By adjusting the free space through the compression ratio and the size of the remaining physical space, the free space can be adjusted to a larger value, reducing the number of dynamic adjustments of the logical space.

[0078] The following describes in detail how to adjust the free space in the logical space according to the remaining physical space with a specific example.

[0079] Figure 3 Another flowchart of a data processing method provided in an embodiment of the present application is as follows: Figure 3As shown, first determine the size of the free space and the remaining physical space. When the logical space is equal to the remaining physical space, all logical spaces are in the free space. When the free space is less than the remaining physical space, add the logical space occupied by the mark to the free space, and then determine the size of the free space and the remaining physical space. If the free space is still less than the remaining physical space, expand the logical space and add the free space until the free space is equal to the remaining physical space. When the free space is greater than the physical space, move the larger part of the free space to the marked occupied space, while the other part is in the free space. The processor is also responsible for recording the actual physical space usage of the current memory with hardware compression. For example, if the physical capacity of the memory with hardware compression is 128GB, the logical space after the operating system is initialized can be 128GB, and all 128GB is in the free space; the logical space can also be 192GB, and 128GB is in the free space and 64GB is in the marked occupied space.

[0080] When data needs to be written to the memory, first apply for a logical space of the corresponding size from the free space and move the applied logical space from the free space to the actual occupied space, then obtain the address mapping table and the physical address at the end of the last write, and send the data, address mapping table, and physical address at the end of the last write to the memory with hardware compression. The memory with hardware compression will compress the data and write it to the memory; in addition, the processor can also receive the compressed length of the data returned by the memory with hardware compression, and update the address mapping table and the physical address after this write. The address mapping table can be obtained from traditional memory or from internal memory. The address mapping table is used to record the corresponding position relationship before and after data compression in the transparent compressed memory. The schematic diagram of the address mapping table is shown as follows: Figure 4 As shown, Figure 4 The size of a logical / physical memory page is 4KB, and the logical starting address is a 4KB page at 0x200000. After the hardware compression of the memory, the actual physical space occupied is located at the page at 0x100000, and the page offset is 0B, and the length is 3KB.

[0081] In this embodiment, by dividing the logical space into occupied space and free space, and dividing the occupied space into actual occupied space and marked occupied space, when the free space is less than the remaining physical space, the marked occupied space is changed to the idle space to increase the free space; on the contrary, when the free space is greater than the remaining physical space, the part of the free space greater than the physical space can be changed to the marked occupied space to reduce the free space. Finally, the effect of dynamically adjusting the free space in the logical space with the remaining physical space is achieved.

[0082] Step 204: write data into the memory according to the adjusted free space.

[0083] In this step, after the free space is adjusted, the compressed data can be written into the memory based on the adjusted free space. Since the free space in the logical space is adjusted based on the actual remaining physical space, it can not only ensure the correct writing of data, but also maximize the use of memory space.

[0084] Exemplarily, in order to ensure that data can be written into the memory correctly, the electronic device also needs to obtain the physical address when the data was written into the memory last time, and update the address mapping table based on the physical address and length information, wherein the address mapping table includes the correspondence between the logical address and the physical address of the data stored in the memory, the logical address is the address in the logical space, and the physical address is the address in the physical space. In this way, when writing data into the memory based on the adjusted free space, the data can be written into the memory based on the free space and the updated address mapping table.

[0085] Specifically, the address in the physical space is the actual address of the memory unit, which is used for chip-level memory unit addressing, and the logical address is often different from the physical address. The correspondence between the physical address and the logical address can be saved using a mapping relationship table. The mapping relationship table can be stored in a memory without hardware compression, such as a memory stick. The memory stick may include DRAM (Dynamic Random Access Memory), or it may be stored in a memory with hardware compression. After obtaining the physical address at the end of the previous data write, the processor can send the physical address at the end of the last write and the cached data to the memory with hardware compression. The memory controller in the memory with hardware compression compresses the cached data through the compression module according to the physical address and writes it to the memory particles. The data this time can continue to be written from the physical address at the end of the last write. Of course, it can also be written by other writing methods. The embodiments of the present application do not limit this. At the same time, after sending the physical address at the end of the last write and the cached data to the memory controller, the processor can also receive the compressed length information of the cached data returned by the memory with hardware compression, specifically, it can receive the compressed length information of the cached data returned by the memory controller; the processor can update the address mapping table through the length information obtained this time and the physical address at the end of the last write, record the correspondence between the logical space and physical space of the data written this time, and update the physical address where the data is written after the end of this write for the next data write, and at the same time update the actual physical space usage of the memory with hardware compression and adjust the free space. For example, Figure 4 In the example, after the data is written, the memory controller compresses 4KB of data into 3KB. After updating the address mapping table, the processor updates the physical address of the next write to (0x100000+3K) and increases the actual physical space usage by 3KB.

[0086] In this embodiment, since the processor can update the address mapping table based on the physical address and length information after the last data was written, the next time data is written, the data can be written to the free space according to the updated address mapping table. This facilitates the storage of data in the memory and ensures the accuracy of data storage.

[0087] The data processing method provided in the embodiment of the present application is that the processor receives the length information of the compressed data sent by the hardware compression device, obtains the remaining physical space of the memory according to the length information, and obtains the free space in the logical space of the memory, and then adjusts the free space according to the remaining physical space so that the difference between the remaining physical space and the adjusted free space is less than the preset value, so that the electronic device can adjust the free space size of the logical space according to the remaining physical space, and achieve the effect that the free space changes with the change of the remaining physical space size. In addition, the electronic device can also write data to the memory according to the size of the free space. Because the processor will also judge the remaining physical space and the logical space size before writing the data to the memory, and adjust the logical space size according to the remaining physical space size, the effect of dynamic adjustment of the free space is achieved, avoiding the data writing error caused by the logical space being too large and the actual physical space being insufficient, and the problem of low memory utilization caused by the logical space being too small and the actual physical space being free, thereby achieving full utilization of the memory space.

[0088] Furthermore, if the data to be processed needs to be read out from the memory, the target logical address of the data to be processed is obtained, and the target physical address corresponding to the target logical address is determined according to the updated address mapping table, the data to be processed is read from the target physical address, and the data to be processed is decompressed, and finally the decompressed data is sent to the cache.

[0089] Specifically, the data to be processed may be cached data, the target logical address is the address of the data to be processed in the logical space, and the target physical address is the address of the data to be processed in the physical space. When data needs to be processed, the target logical address of the data to be processed is first obtained, and the target physical address corresponding to the target logical address can be found according to the address mapping table. The processor can send the obtained target physical address to the memory controller, and then receive the decompressed data returned by the decompression module and store it in the cache. Among them, the reading of data can be implemented using the memory controller, and the decompression of data can be implemented using the decompression module.

[0090] In this embodiment, by combining the target logical space address and the address mapping table, the physical space address of the data to be processed can be obtained. The physical space address can be used to read the data to be processed, and further decompress the data to be processed and cache the decompressed data.

[0091] In a possible implementation, when data is deleted from the memory, the physical space and the corresponding logical space in the memory will be in a released state. At this time, in order to make use of this part of the released space, the physical space and the logical space in the released state will be considered when adjusting the free space. Exemplarily, when part of the logical space in the logical space is in a released state, the free space is updated according to the part of the logical space, and the physical space corresponding to the part of the logical space is determined according to the address mapping table, and then the remaining physical space is updated according to the physical space corresponding to the part of the logical space. In this way, when the free space is adjusted according to the remaining physical space, the updated free space can be adjusted according to the updated remaining physical space.

[0092] Specifically, part of the logical space is released, that is, part of the logical space is changed from an occupied state to a non-occupied state. The release of part of the logical space may be caused by operations such as data in the logical space being moved or deleted. After part of the logical space is released, the processor can find the corresponding physical space according to the address mapping table, and mark the corresponding physical space found as part of the remaining physical space, and then update the total remaining physical space by adding it to the existing remaining physical space. After updating the remaining physical space, the free space can also be adjusted according to the first difference between the free space and the remaining physical space. The processor can also directly add a part of the logical space to the free space after a part of the logical space is released, and at the same time obtain the address mapping table, and add the physical space corresponding to the part of the logical space to the remaining physical space according to the address mapping table.

[0093] In this embodiment, since the processor can adjust the remaining physical space and free space according to the released logical space, timely adjustment of the physical space and free space can be achieved. When new data needs to be stored, the adjusted remaining physical space and free space can be stored, so that the released space can be fully utilized.

[0094] Figure 5 A schematic diagram of the structure of a data processing device 50 provided in an embodiment of the present application, for example, see Figure 5 As shown, the data processing device 50 may include: a receiving module 501 , an acquiring module 502 and a processing module 503 .

[0095] The receiving module 501 is used to receive the length information of the compressed data sent by the hardware compression device;

[0096] An acquisition module 502 is used to acquire the remaining physical space of the memory according to the length information;

[0097] The acquisition module 502 is also used to acquire free space in the logical space of the memory;

[0098] The processing module 503 is used to adjust the free space according to the remaining physical space;

[0099] The processing module 503 is further configured to write data into the memory according to the adjusted free space.

[0100] In a possible implementation, the processing module 503 is specifically configured to:

[0101] A first difference between the remaining physical space and the free space is determined, and the free space is adjusted according to the first difference.

[0102] In a possible implementation, the processing module 503 is specifically configured to:

[0103] Determine whether the remaining physical space is greater than the free space;

[0104] If the remaining physical space is greater than the free space, the free space is expanded according to the first difference;

[0105] If the remaining physical space is less than the free space, the free space is reduced according to the first difference.

[0106] In a possible implementation, the acquisition module 502 is further used to: acquire the physical address when the last writing of data into the memory is completed.

[0107] The processing module 503 is further used to: update the address mapping table according to the physical address and length information, the address mapping table including the correspondence between the logical address and the physical address of the data stored in the memory, the logical address is the address in the logical space, and the physical address is the address in the physical space;

[0108] The processing module 503 is specifically used for:

[0109] Write data to the memory based on the free space and the updated address mapping table.

[0110] In a possible implementation, the processing module 503 is specifically configured to: when part of the logical space is in a released state, update the free space according to the part of the logical space;

[0111] Determine the physical space corresponding to the portion of the logical space according to the address mapping table;

[0112] Update the remaining physical space according to the physical space corresponding to the part of the logical space;

[0113] Adjust the updated free space according to the updated remaining physical space.

[0114] In a possible implementation, the acquisition module 502 is further configured to: acquire a target logical address of the data to be processed.

[0115] The processing module 503 is further used to: obtain a target logical address of the data to be processed;

[0116] Determine the target physical address corresponding to the target logical address according to the updated address mapping table;

[0117] Read the data to be processed from the target physical address and decompress the data to be processed;

[0118] Send the decompressed data to the cache.

[0119] Figure 6 A schematic diagram of the structure of a data processing device 60 provided in an embodiment of the present application. For example, see Figure 6 As shown, the electronic device 60 may include a processor 601 and a memory 602, wherein the memory further includes a memory controller and a memory. The memory controller includes a compression module and a decompression module.

[0120] The memory 602 is used to store computer programs, receive processor instructions, and compress and decompress data according to the processor instructions.

[0121] The processor 601 is used to read the computer program stored in the memory 602 and execute the technical solution of the data processing method in any of the above embodiments according to the computer program in the memory 602.

[0122] The present application also provides a computer-readable storage medium, which stores computer execution instructions. When the processor executes the computer execution instructions, the technical solution of the data processing method in any of the above-mentioned embodiments is implemented. Its implementation principle and beneficial effects are similar to the implementation principle and beneficial effects of the data processing method. Please refer to the implementation principle and beneficial effects of the data processing method, which will not be repeated here.

[0123] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the technical solution of the data processing method in any of the above-mentioned embodiments. Its implementation principle and beneficial effects are similar to those of the data processing method. Please refer to the implementation principle and beneficial effects of the data processing method, and no further details will be given here.

[0124] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0125] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: include: Receive the length information of the compressed data sent by the hardware compression device, and obtain the remaining physical space of the memory according to the length information; Acquire free space in the logical space of the memory; adjusting the free space according to the remaining physical space; Writing data into the memory according to the adjusted free space; The step of adjusting the free space according to the remaining physical space includes: Determine whether the remaining physical space is greater than the free space; if the remaining physical space is greater than the free space If there is free space, the logical space occupied by the tag is added to the free space; if the updated free space is smaller than the remaining physical space, the logical space is expanded and added to the updated free space until the updated free space is equal to the remaining physical space; if the remaining physical space is smaller than the free space, part of the free space larger than the remaining physical space is moved to the space occupied by the tag.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining the physical address at the end of the last writing of data into the memory; According to the physical address and the length information, an address mapping table is updated, wherein the address mapping table includes a correspondence between a logical address and a physical address of the data stored in the memory, wherein the logical address is an address in the logical space, and the physical address is an address in the physical space; Writing data into the memory according to the adjusted free space includes: Data is written into the memory according to the free space and the updated address mapping table.

3. The method according to claim 1, characterized in that The method further comprises: When part of the logical space is in a released state, updating the free space according to the part of the logical space; Determine, according to the address mapping table, a physical space corresponding to the portion of the logical space; updating the remaining physical space according to the physical space corresponding to the part of the logical space; The step of adjusting the free space according to the remaining physical space includes: Adjust the updated free space according to the updated remaining physical space.

4. The method according to claim 2, characterized in that: The method further comprises: Obtaining the target logical address of the data to be processed; Determining a target physical address corresponding to the target logical address according to the updated address mapping table; Reading the data to be processed from the target physical address, and decompressing the data to be processed; Send the decompressed data to the cache.

5. A data processing device, characterized in that: include: A receiving module, used for receiving the length information of the compressed data sent by the hardware compression device; An acquisition module, used for acquiring the remaining physical space of the memory according to the length information; The acquisition module is further used to acquire free space in the logical space of the memory; A processing module, configured to adjust the free space according to the remaining physical space so that a difference between the remaining physical space and the adjusted free space is less than a preset value; The processing module is further used to write data into the memory according to the adjusted free space; The processing module is specifically used to determine whether the remaining physical space is greater than the free space; if the remaining physical space is greater than the free space, then add the logical space occupied by the mark to the free space; if the updated free space is less than the remaining physical space, then expand the logical space and add it to the updated free space until the updated free space is equal to the remaining physical space; If the remaining physical space is smaller than the free space, a portion of the free space that is larger than the remaining physical space is moved to the mark occupied space.

6. A data processing device comprising: Processor, memory; Memory for storing computer-executable instructions; The processor is used to execute the computer-executable instructions stored in the memory to implement the data processing method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the data processing method according to any one of claims 1 to 4 when executed by a processor.

8. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images