Data processing method and device, equipment, storage medium and program product

Abstract

The invention provides a data processing method and device, equipment, a storage medium and a program product. The method is applied to expansion equipment of main equipment, the expansion equipment comprises a compression engine and an expansion memory, and the method comprises the steps that a data storage request sent by the main equipment is received, and the data storage request comprises first initial data; performing compression processing on the first initial data through a compression engine to obtain first compressed data; and storing the first compressed data in the extended memory. And the data storage efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a data processing method, apparatus, device, storage medium, and program product. Background Art

[0002] The electronic device may be provided with a memory for storing data in the memory. When the memory in the electronic device is insufficient, the memory may be expanded via a bus connection to expand the memory used by the electronic device.

[0003] In related art, when initial data in the main memory needs to be stored in the extended memory, the initial data can be copied from the main memory to a compression accelerator, which then compresses the initial data to obtain compressed data. The compressed data is then copied to the main memory, and finally the compressed data in the main memory is copied to the extended memory. However, in this process, the steps for transferring the initial data from the main memory to the extended memory are complex, resulting in low data storage efficiency. Summary of the Invention

[0004] Various aspects of the present application provide a data processing method, apparatus, device, storage medium, and program product to improve the efficiency of storing data.

[0005] In a first aspect, an embodiment of the present application provides a data processing method applied to an expansion device of a master device, the expansion device including a compression engine and an expansion memory, the method comprising:

[0006] receiving a data storage request sent by the master device, wherein the data storage request includes first initial data;

[0007] compressing the first initial data by the compression engine to obtain first compressed data;

[0008] The first compressed data is stored in the extended memory.

[0009] In a possible implementation, the expansion device further includes a temporary cache, and the data storage request is used to request writing the first initial data into the temporary cache; and compressing the first initial data by the compression engine to obtain the first compressed data includes:

[0010] When the master device finishes writing the first initial data into the temporary buffer, obtaining the first initial data from the temporary buffer by the compression engine;

[0011] The first initial data is compressed to obtain the first compressed data.

[0012] In a possible implementation, multiple virtual address spaces are provided in the expansion device, wherein the first initial data is written by the main device into the temporary cache through a first virtual address space, and the first virtual address space is an idle virtual address space among the multiple virtual address spaces.

[0013] In a possible implementation manner, after storing the first compressed data in the extended memory, the method further includes:

[0014] determining a first actual storage space of the first compressed data in the extended memory;

[0015] A mapping relationship between the first actual storage space and the first virtual address space is determined, and the mapping relationship is stored.

[0016] In one possible implementation, the method further includes:

[0017] After acquiring the first initial data in the temporary buffer by the compression engine, sending a data storage response to the master device;

[0018] The data storage response is used to indicate completion of the first initial data storage, and the data storage response is used to enable the main device to asynchronously store other data to the expansion device.

[0019] In a possible implementation, after obtaining the first initial data from the temporary cache by the compression engine, the method further includes:

[0020] The first initial data is deleted from the temporary buffer.

[0021] In one possible implementation, the method further includes:

[0022] receiving a data read request sent by the master device;

[0023] acquiring second compressed data in the extended memory according to the data read request;

[0024] The second compressed data is decompressed by the compression engine to obtain second initial data, and the second initial data is synchronized to the master device.

[0025] In a possible implementation, the data read request includes an identifier of the second virtual address space; and obtaining the second compressed data in the extended memory according to the data read request includes:

[0026] determining a second actual storage space according to the identifier of the second virtual address space;

[0027] The second compressed data is obtained in the extended memory according to the second actual storage space.

[0028] In a possible implementation, synchronizing the second initial data to the master device includes:

[0029] storing the second initial data in a temporary buffer of the expansion device;

[0030] A data read response is sent to the master device, where the data read response is used to instruct the master device to read the second initial data in the temporary cache through the second virtual address space.

[0031] In a second aspect, an embodiment of the present application provides a data processing device applied to an expansion device of a main device, wherein the expansion device includes a compression engine and an expansion memory, and the device includes: a receiving module, a processing module, and a storage module, wherein:

[0032] The receiving module is configured to receive a data storage request sent by the master device, wherein the data storage request includes first initial data;

[0033] The processing module is configured to compress the first initial data through the compression engine to obtain first compressed data;

[0034] The storage module is used to store the first compressed data in the extended memory.

[0035] In a possible implementation, the expansion device further includes a temporary cache, and the data storage request is used to request writing the first initial data into the temporary cache; and the processing module is specifically configured to:

[0036] When the master device finishes writing the first initial data into the temporary buffer, obtaining the first initial data from the temporary buffer by the compression engine;

[0037] The first initial data is compressed to obtain the first compressed data.

[0038] In a possible implementation, multiple virtual address spaces are provided in the expansion device, wherein the first initial data is written by the main device into the temporary cache through a first virtual address space, and the first virtual address space is an idle virtual address space among the multiple virtual address spaces.

[0039] In a possible implementation, after storing the first compressed data in the extended memory, the device further includes a determination module, wherein:

[0040] The determining module is used to determine a first actual storage space of the first compressed data in the extended memory;

[0041] The determining module is further configured to determine a mapping relationship between the first actual storage space and the first virtual address space;

[0042] The storage module is further configured to store the mapping relationship.

[0043] In a possible implementation manner, the device further includes: a sending module,

[0044] The sending module is configured to send a data storage response to the master device after acquiring the first initial data in the temporary buffer through the compression engine;

[0045] The data storage response is used to indicate completion of the first initial data storage, and the data storage response is used to enable the main device to asynchronously store other data to the expansion device.

[0046] In a possible implementation, after the compression engine obtains the first initial data from the temporary cache, the apparatus further includes a deletion module:

[0047] The deleting module is configured to delete the first initial data in the temporary cache.

[0048] In a possible implementation, the device further includes an acquisition module and a synchronization module, wherein:

[0049] The receiving module is further configured to receive a data read request sent by the master device;

[0050] The acquisition module is configured to acquire the second compressed data from the extended memory according to the data read request;

[0051] The processing module is configured to decompress the second compressed data through the compression engine to obtain second initial data;

[0052] The synchronization module is further configured to synchronize the second initial data to the master device.

[0053] In a possible implementation, the data read request includes an identifier of the second virtual address space; and the acquisition module is specifically configured to:

[0054] determining a second actual storage space according to the identifier of the second virtual address space;

[0055] The second compressed data is obtained in the extended memory according to the second actual storage space.

[0056] In a possible implementation, the synchronization module is specifically configured to:

[0057] storing the second initial data in a temporary buffer of the expansion device;

[0058] A data read response is sent to the master device, where the data read response is used to instruct the master device to read the second initial data in the temporary cache through the second virtual address space.

[0059] In a third aspect, an embodiment of the present application provides an expansion device, including:

[0060] at least one processor; and

[0061] a memory, a compression engine, and an expansion memory communicatively coupled to the at least one processor;

[0062] The compression engine is used to compress and / or decompress data;

[0063] The extended memory is used to store data;

[0064] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the expansion device to execute any one of the methods described in the first aspect.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which are used to implement the method described in any one of the first aspects when the computer-executable instructions are executed by a processor.

[0066] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the method shown in any one of the first aspects when executed by a processor.

[0067] Embodiments of the present application provide a data processing method, apparatus, device, storage medium, and program product. An expansion device can receive a data storage request from a master device and, using a compression engine, compress first initial data to obtain first compressed data, which can then be stored in an expanded memory. Because the expansion device can both compress the first initial data using the compression engine and store the first compressed data in the expanded memory, compared to existing technologies, only one data copy is required between the master device and the expansion device, reducing interaction between the master device and the expansion device, thereby improving data storage efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0069] Figure 1 A schematic diagram of a scenario provided for an exemplary embodiment of the present application;

[0070] Figure 2 A schematic diagram of a data processing method in related art;

[0071] Figure 3 A flowchart of a data processing method provided by an exemplary embodiment of the present application;

[0072] Figure 4 A flowchart of another data processing method provided for an exemplary embodiment of the present application;

[0073] Figure 5 A process diagram of a data processing method provided by an exemplary embodiment of the present application;

[0074] Figure 6 A flowchart of another data processing method provided by an exemplary embodiment of the present application;

[0075] Figure 7 A process diagram of another data processing method provided by an exemplary embodiment of the present application;

[0076] Figure 8 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0077] Figure 9 A schematic diagram of the structure of another data processing device provided in an embodiment of the present application;

[0078] Figure 10 A schematic structural diagram of an expansion device provided in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0079] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0080] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0081] Figure 1 This is a schematic diagram of a scenario provided by an exemplary embodiment of this application. Figure 1 , including main equipment and expansion equipment.

[0082] The main device may include initial data, and the expansion device may compress the initial data in the main device to obtain compressed data and store the compressed data.

[0083] In related art, when initial data in the main memory needs to be stored in the extended memory, the initial data can be copied from the main memory to a compression accelerator, which then compresses the initial data to obtain compressed data. The compressed data is then copied to the main memory, and finally the compressed data in the main memory is copied to the extended memory. However, in this process, the steps for transferring the initial data from the main memory to the extended memory are complex, resulting in low data storage efficiency.

[0084] In an embodiment of the present application, the expansion device may include a compression engine and an expansion memory. The expansion device may compress the first initial data in the main device using the compression engine to obtain first compressed data, and store the first compressed data in the expansion memory. Because the expansion device can both compress the first initial data using the compression engine and store the first compressed data in the expansion memory, compared to existing technologies, only one data copy is required between the main device and the expansion device, reducing the interaction between the main device and the expansion device, thereby improving data storage efficiency.

[0085] Next, combine Figure 2 , the data processing method in the related technology is explained.

[0086] Figure 2 This is a schematic diagram of a data processing method in related technology. Figure 2 , including the main device, extended memory, and compression accelerator.

[0087] The main device may include a main memory and a processor (Central Processing Unit, CPU). The main memory stores initial data.

[0088] The compression accelerator can be used to compress data. The extended memory can be used to store data.

[0089] In step 1, the CPU can send a compression instruction to the compression accelerator via the Peripheral Component Interconnect (PCI) bus or the Memory-Mapped Input / Output (MMIO) bus, instructing the compression accelerator to perform the compression task. The compression instruction can include the corresponding starting address and memory length of the specified memory, the compression algorithm, and the compression granularity.

[0090] In step ②, the compression accelerator may read the initial data from the host device through direct memory access (DMA) technology according to the compression instruction, and copy the initial data to the compression accelerator.

[0091] In step ③, the compression accelerator may perform compression processing on the initial data to obtain compressed data.

[0092] In step ④, the compression accelerator may copy the compressed data to the main memory through DMA, so as to store the compressed data in the main memory.

[0093] In step ⑤, the CPU may transfer the compressed data in the main memory to the extended memory via the bus, so as to store the compressed data in the extended memory.

[0094] exist Figure 2 In the related art shown, the main device, compression accelerator, and extended memory are three independent devices. This requires two data copies between the main device and the compression accelerator, and one data copy between the main device and the extended memory, for a total of three data copies, before the compressed data can be stored in the extended memory. This results in low data storage efficiency.

[0095] The technical solutions shown in this application are described in detail below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination with each other, and the same or similar contents will not be repeated in different embodiments.

[0096] Figure 3 This is a flow chart of a data processing method provided by an exemplary embodiment of the present application. Figure 3 , the method may include:

[0097] S301: Receive a data storage request sent by a master device.

[0098] The execution subject of the embodiments of the present application may be an expansion device or a data processing device within the expansion device. The data processing device may be implemented through software or a combination of software and hardware. For ease of understanding, the following description will be based on an example in which the execution subject is an expansion device.

[0099] The main device may include a main memory and a CPU. The main memory may store first initial data.

[0100] A main device can have a corresponding expansion device. This expansion device can include a compression engine and extended memory. The compression engine can be used to compress data; the extended memory can be used to store data. The extended memory and main memory can be different types of memory.

[0101] The data storage request may include the first initial data. The data storage request is used to request writing the first initial data into the expansion device.

[0102] The size of the first initial data is the same as the size of a memory page. The size of a memory page can be any of the following: 4KB (kilobytes), 16KB, 2M (megabytes), 1G (gigabytes), etc.

[0103] For example, if the main memory includes multiple memory pages and the size of each memory page is 4 KB, the size of the first initial data may be 4 KB.

[0104] The main device may send a data storage request to the expansion device, so that the expansion device receives the data storage request sent by the main device.

[0105] For example, the expansion device may receive a data storage request 1 sent by the main device, where the data storage request 1 may include initial data 1 with a size of 4 KB.

[0106] S302: Compress the first initial data using a compression engine to obtain first compressed data.

[0107] Optionally, the extension device may further include a temporary buffer.

[0108] Because the expansion device also includes a temporary buffer, the data storage request can be used to request writing the first initial data into the temporary buffer. The size of the temporary buffer is the same as the size of the first initial data, that is, the size of the temporary buffer is also the same as the size of a memory page in the main memory. For example, if the size of a memory page is 4 KB, the size of the temporary buffer can be 4 KB.

[0109] In an optional embodiment, the first initial data can be compressed by a compression engine to obtain first compressed data in the following manner: when the master device finishes writing the first initial data in the temporary cache, the compression engine obtains the first initial data in the temporary cache; and compresses the first initial data to obtain first compressed data.

[0110] The master device may write the first initial data in the data storage request into a temporary buffer of the expansion device. When the master device finishes writing the first initial data into the temporary buffer, the expansion device may obtain the first initial data from the temporary buffer using a compression engine, and compress the first initial data using the compression engine to obtain first compressed data.

[0111] For example, if the data storage request 1 includes initial data 1 of size 4KB, the master device can write the initial data 1 in the temporary cache. When the master device finishes writing the initial data 1 in the temporary cache, the expansion device can obtain the initial data 1 in the temporary cache through the compression engine and compress the initial data 1 to obtain compressed data 1. Assume that the size of the compressed data 1 is 2KB.

[0112] S303: Store the first compressed data in the extended memory.

[0113] Optionally, the extended memory in the extended device and the main memory in the main device may be different types of memory.

[0114] Optionally, the expansion device may determine a first actual storage space in the expansion memory through a memory allocation function (malloc) according to the size of the first compressed data, and store the first compressed data in the first actual storage space.

[0115] The size of the first actual storage space is the same as the size of the first compressed data.

[0116] For example, if the size of the compressed data 1 is 2 KB, the expansion device may determine 2 KB of actual storage space 1 in the expansion memory through the malloc function, and store 2 KB of the compressed data 1 in the actual storage space 1 .

[0117] In an embodiment of the present application, the expansion device can receive a data storage request sent by the master device and, using a compression engine, compress the first initial data to obtain first compressed data, which can then be stored in the expanded memory. Because the expansion device can both compress the first initial data using the compression engine and store the first compressed data in the expanded memory, compared to existing technologies, only one data copy is required between the master device and the expansion device, reducing interaction between the master device and the expansion device, thereby improving data storage efficiency.

[0118] The technical solution of this application may include two processes, namely, the process of storing data in the expansion device and the process of reading data in the expansion device. Figure 4-Figure 5 , describes in detail the process of storing data in the expansion device; combined with Figure 6-Figure 7 , which details the process of reading data from the expansion device.

[0119] Figure 4 A flowchart of another data processing method provided by an exemplary embodiment of the present application. Figure 4 , the method may include:

[0120] S401: Receive a data storage request sent by a master device.

[0121] It should be noted that the execution process of step S401 can refer to step S301 and will not be repeated here.

[0122] S402 : When the master device finishes writing the first initial data into the temporary buffer, the compression engine obtains the first initial data from the temporary buffer.

[0123] Optionally, the expansion device may be provided with multiple virtual address spaces. The size of each virtual address space is the same. The size of any virtual address space is the same as the size of the first initial data, that is, the size of any virtual address space is the same as the size of a memory page in the main memory. For example, if the size of a memory page is 4 KB, the size of any virtual address space can be 4 KB.

[0124] Alternatively, the multiple virtual address spaces may be continuous or discontinuous.

[0125] It is important to note that since the size of the temporary cache is the same as the size of a memory page in main memory, and the size of any virtual address space is the same as the size of a memory page in main memory, the size of the temporary cache is the same as the size of the first virtual address space. For example, if the size of a memory page is 4KB, the size of the temporary cache can be 4KB, and the size of any virtual address space can also be 4KB.

[0126] The data storage request may be used to write first initial data in the temporary buffer.

[0127] Optionally, the data storage request may further include an identifier of the first virtual address space. Optionally, the identifier of the first virtual address space may be the starting address of the first virtual address space. For example, if the first virtual address space is virtual address space 1, the identifier of virtual address space 1 may be address A, where address A is the starting address of virtual address space 1.

[0128] The data storage request may also be used to request writing of first initial data in the temporary cache through the first virtual address space.

[0129] The first initial data may be written into a temporary cache for the master device through a first virtual address space, where the first virtual address space is an idle virtual address space among the multiple virtual address spaces.

[0130] The master device may write the first initial data into the temporary buffer of the extension device in a sequential manner through the first virtual address space. After the last byte of the first virtual address space is written into the temporary buffer, the extension device may determine that the master device has finished writing the first initial data into the temporary buffer. The extension device may obtain the first initial data from the temporary buffer through the compression engine.

[0131] For example, if the first virtual address space is virtual address space 1, the main device can write 4K initial data 1 to the temporary cache through virtual address space 1, until the data is written to the temporary cache through the last byte of virtual address space 1, and the expansion device can obtain the 4K initial data 1 in the temporary cache through the compression engine.

[0132] S403: After obtaining the first initial data in the temporary buffer through the compression engine, a data storage response is sent to the master device.

[0133] The data storage response is used to indicate completion of the first initial data storage, and the data storage response is used to enable the main device to asynchronously store other data to the expansion device.

[0134] For example, after the compression engine obtains 4KB of initial data 1 in the temporary cache, the expansion device may send a data storage response 1 to the main device, so that the main device may asynchronously store the 4KB of first initial data 2 to the expansion device.

[0135] S404: After the compression engine obtains the first initial data from the temporary cache, the first initial data is deleted from the temporary cache.

[0136] Optionally, after the expansion device obtains data from the temporary cache through the compression engine, the first initial data may be deleted from the temporary cache so that the temporary cache can be used to store other data that the main device stores to the expansion device.

[0137] For example, after the compression engine obtains 4KB of initial data 1 in the temporary cache, the expansion device can delete the 4KB of initial data 1 in the temporary cache so that the main device can asynchronously write 4KB of first initial data 2 to the temporary cache.

[0138] It should be noted that step S403 and step S404 may be executed in any order or simultaneously.

[0139] S405: Compress the first initial data to obtain first compressed data.

[0140] For example, the expansion device may compress the 4KB initial data 1 through the compression engine to obtain the compressed data 1. Assume that the size of the compressed data 1 is 2KB.

[0141] S406: Store the first compressed data in the extended memory.

[0142] It should be noted that the execution process of step S406 can refer to step S303 and will not be repeated here.

[0143] S407: Determine a first actual storage space of the first compressed data in the extended memory.

[0144] Alternatively, the first actual storage space may be represented by the starting address and storage length of the first actual storage space. For example, if the first actual storage space is actual storage space 1, the starting address of actual storage space 1 is address B, and the storage length is 2 KB, then actual storage space 1 may be represented as address B + storage length 2 KB.

[0145] For example, the extended memory device may determine that the first actual storage space of the compressed data 1 in the extended memory is actual storage space 1. Assume that the actual storage space 1 may be determined to be address B+storage length 2KB.

[0146] S408: Determine a mapping relationship between the first actual storage space and the first virtual address space, and store the mapping relationship.

[0147] Because the main device stores the first initial data in the expansion device via the first virtual address space, and the expansion device compresses the first initial data using a compression engine to obtain the first compressed data, and then stores the first compressed data in the first actual storage space of the expansion memory, there is a one-to-one correspondence between the first virtual address space and the first actual storage space. The expansion device can determine the mapping relationship between the first actual storage space and the first virtual address space and store the mapping relationship.

[0148] For example, if the first virtual address space is virtual address space 1, the first actual storage space is actual storage space 1, and actual storage space 1 can be expressed as address B + storage length 2KB, then the expansion device can determine mapping relationship 1 as: virtual address space 1 corresponds to actual storage space 1, that is, address B + storage length 2KB, and store the mapping relationship 1.

[0149] Optionally, the mapping relationship may include identifiers of multiple virtual address spaces and identifiers of actual storage spaces corresponding to the identifiers of each virtual address space. The identifier of the virtual address space may be represented by the starting address of the virtual address space, and the identifier of the actual storage space may be represented by the starting address and storage length of the actual storage space.

[0150] For example, the mapping relationship may be as shown in Table 1:

[0151] Table 1

[0152] Virtual address space identifier Identification of actual storage space Virtual address space 1 (address A) Actual storage space 1 (address B + storage length 2KB) Virtual address space 2 (address C) Actual storage space 2 (address D + storage length 1) …… ……

[0153] In the technical solution of the present application, the expansion device can organically combine the compression engine with the extended memory, that is, the memory expansion of the main memory in the main device is realized through the extended memory, and the data is compressed by the compression engine to improve storage efficiency.

[0154] It should be noted that the process of the main device writing the initial data into the temporary cache of the expansion device and the process of the expansion device compressing and storing the initial data can be performed asynchronously, which can further improve the efficiency of the main device in storing data in the expansion device.

[0155] If the primary device needs to read the initial data while the expansion device is compressing and storing it, the primary device's read request will be blocked until the initial data is stored. However, this situation is unlikely to occur because expansion devices are primarily used to store data that is not frequently used by the primary device, so this situation can be safely avoided.

[0156] In an embodiment of the present application, the expansion device can receive a data storage request sent by the main device, and when the main device finishes writing the first initial data in the temporary cache, obtain the first initial data in the temporary cache through the compression engine. After the expansion device obtains the first initial data in the temporary cache through the compression engine, it can send a data storage response to the main device, and can also delete the first initial data in the temporary cache. The expansion device can compress the first initial data to obtain first compressed data, and store the first compressed data in the extended memory. The expansion device can also determine the first actual storage space of the first compressed data in the extended memory, and determine the mapping relationship between the first actual storage space and the first virtual address space, and store the mapping relationship. Since the expansion device can both compress the first initial data through the compression engine and store the first compressed data in the extended memory, compared with the existing technology, only one data copy is required between the main device and the expansion device, which reduces the interaction between the main device and the expansion device, thereby improving the efficiency of data storage.

[0157] It should be noted that in Figure 4The various processing steps (S401 to S408) shown in the embodiment do not constitute a specific limitation on the data processing process. In other embodiments of the present application, the data processing process may include Figure 4 The embodiments may include more or fewer steps. For example, the data processing process may include Figure 4 Some steps in the embodiment, or Figure 4 Some steps in the embodiment may be replaced by steps with the same functions, or Figure 4 Some steps in the embodiments may be split into multiple steps, etc.

[0158] Below, in Figure 4 Based on the embodiment shown, combined Figure 5 , the above process of storing data in the expansion device is further explained through specific examples.

[0159] Figure 5 This is a process diagram of a data processing method provided by an exemplary embodiment of the present application. Figure 5 , including main equipment and expansion equipment.

[0160] The main device may include a main memory and a CPU. The main memory may include multiple memory pages. Among them, memory page 1 may store 4KB of initial data 1.

[0161] The expansion device may include temporary buffers, compression engines, and extended memory.

[0162] In step ①, the master device may send a data storage request 1 to the expansion device. The data storage request 1 may include 4 KB of initial data 1. The data storage request may also include address A, which is the starting address of virtual address space 1. The data storage request 1 may be used to request that the 4 KB of initial data 1 be written to the temporary cache via virtual address space 1.

[0163] In step ②, when the main device finishes writing the 4KB initial data 1 into the temporary cache, the expansion device can obtain the 4KB initial data 1 from the temporary cache through the compression engine.

[0164] Optionally, after the main device obtains 4KB of initial data 1 in the temporary cache through the compression engine, the expansion device can send a data storage response 1 to the main device, so that the main device can asynchronously store the 4KB of initial data 2 to the expansion device; the expansion device can also delete the 4KB of initial data 1 in the temporary cache, so that the main device can asynchronously write the 4KB of initial data 2 to the temporary cache.

[0165] In step ③, the expansion device may compress the 4KB initial data 1 through the compression engine to obtain 2KB compressed data 1.

[0166] In step ④, the expansion device may determine 2 KB of actual storage space 1 in the expansion memory, and store 2 KB of compressed data 1 in the actual storage space 1 .

[0167] Optionally, the expansion device may also determine a mapping relationship between the actual storage space 1 and the virtual address space 1, and store the mapping relationship.

[0168] In an embodiment of the present application, the expansion device can receive a data storage request sent by the main device, and when the main device finishes writing the first initial data in the temporary cache, the expansion device can obtain the first initial data in the temporary cache through the compression engine. After the expansion device obtains the first initial data in the temporary cache through the compression engine, it can send a data storage response to the main device, and can also delete the first initial data in the temporary cache. The expansion device can compress the first initial data to obtain first compressed data, and store the first compressed data in the extended memory. The expansion device can also determine the first actual storage space of the first compressed data in the extended memory, and determine the mapping relationship between the first actual storage space and the first virtual address space, and store the mapping relationship. Since the expansion device can both compress the first initial data through the compression engine and store the first compressed data in the extended memory, compared with the existing technology, only one data copy is required between the main device and the expansion device, which reduces the interaction between the main device and the expansion device, thereby improving the efficiency of data storage.

[0169] Figure 6 A flowchart of another data processing method provided by an exemplary embodiment of the present application. Figure 6 , the method may include:

[0170] S601: Receive a data read request sent by a master device.

[0171] The data read request may include the second virtual address space. The data read request may be used to obtain initial data from a temporary buffer of the expansion device through the second virtual address space.

[0172] Optionally, the identifier of the second virtual address space may be a starting address of the second virtual address space.

[0173] The main device can send a data read request to the extension device, and the extension device can receive the data read request sent by the main device.

[0174] For example, the master device may send a data read request 1 to the expansion device. If the second virtual address space is virtual address space 1, the data read request 1 may include address A, which may be the starting address of virtual address space 1. The expansion device may receive the data read request 1 sent by the master device.

[0175] S602: Obtain second compressed data from the extended memory according to the data read request.

[0176] In an optional embodiment, the second compressed data may be obtained in the extended memory according to the data read request in the following manner: determining the second actual storage space according to the second virtual address space; and obtaining the second compressed data in the extended memory according to the second actual storage space.

[0177] Since there is a mapping relationship between the virtual address space and the actual storage space, the expansion device can determine the identifier of the second actual storage space according to the identifier of the second virtual address space in the mapping relationship, and determine the second actual storage space according to the identifier of the second actual storage space.

[0178] For example, if data read request 1 includes address A, which is the starting address of virtual address space 1, and if the mapping relationship is as shown in Table 1, the expansion device can determine, based on address A, that the corresponding second actual storage space is actual storage space 1. Actual storage space 1 is identified by address B + storage length 2 KB. The expansion device can then determine actual storage space 1 in the extended memory based on address B + storage length 2 KB. The expansion device can obtain the second compressed data from actual storage space 1. The second compressed data may be compressed data 1.

[0179] S603: Decompress the second compressed data through the compression engine to obtain second initial data, and synchronize the second initial data to the master device.

[0180] In an optional embodiment, the second initial data may be synchronized to the master device in the following manner: storing the second initial data in a temporary buffer of the expansion device; and sending a data read response to the master device.

[0181] The data read response may be used to instruct the master device to read the second initial data in the temporary cache through the second virtual address space.

[0182] For example, if the second compressed data is compressed data 1, the expansion device may decompress compressed data 1 using the compression engine to obtain second initial data. Assume that the second initial data is initial data 1. The expansion device may store initial data 1 in a temporary cache and send a data read response 1 to the master device. If the second virtual address space is virtual address space 1, the data read response 1 may be used to instruct the master device to sequentially read initial data 1 from the temporary cache via virtual address space 1.

[0183] It should be noted that, in the technical solution of the present application, when the expansion device performs compression and decompression processing through the compression engine, it can be performed at the same data granularity.

[0184] In an embodiment of the present application, the expansion device can receive a data read request sent by the main device and, based on the data read request, obtain the second compressed data in the expansion memory. The expansion device can then decompress the second compressed data using a compression engine to obtain the second initial data, and synchronize the second initial data to the main device. Because the expansion device can both store the second compressed data in the expansion memory and decompress the second compressed data using a compression engine to obtain the second initial data, the main device can obtain the second initial data from the expansion device through a single data copy. Compared to existing technologies, only one data copy is required between the main device and the expansion device, reducing the interaction between the main device and the expansion device, thereby improving the efficiency of data acquisition.

[0185] Below, in Figure 6 Based on the embodiment shown, combined Figure 7 , through specific examples, the above process of reading data in the expansion device is further explained.

[0186] Figure 7 A schematic diagram of another data processing method provided by an exemplary embodiment of the present application. Figure 7 , including main equipment and expansion equipment.

[0187] The main device may include a main memory and a CPU. The main memory may include multiple memory pages.

[0188] The expansion device may include temporary buffers, compression engines, and extended memory.

[0189] In step ①, the master device may send a data read request 1 to the expansion device. The data read request 1 may include address A, which is the starting address of virtual address space 1. The data storage request 1 may be used to request to obtain initial data from the temporary cache via virtual address space 1.

[0190] In step 2, the expansion device can determine, based on address A in the mapping relationship, that the corresponding second actual storage space is actual storage space 1. Actual storage space 1 is identified by address B + storage length 2 KB. The expansion device can then determine actual storage space 1 in the extended memory based on address B + storage length 2 KB. The expansion device can then obtain 2 KB of compressed data 1 from actual storage space 1.

[0191] In step ③, the expansion device may decompress the 2KB compressed data 1 through the compression engine to obtain the 4KB initial data 1.

[0192] In step ④, the expansion device may store the 4 KB initial data 1 in a temporary cache.

[0193] In step ⑤, the expansion device may send a data read request 1 to the main device, so that the main device reads the initial data 1 in the temporary cache through the virtual address space 1.

[0194] The master device may obtain 4KB of initial data 1 in the temporary cache and store the initial data 1 in memory page 1 of the main memory.

[0195] In an embodiment of the present application, the expansion device can receive a data read request sent by the main device and, based on the data read request, obtain the second compressed data in the expansion memory. The expansion device can then decompress the second compressed data using a compression engine to obtain the second initial data, and synchronize the second initial data to the main device. Because the expansion device can both store the second compressed data in the expansion memory and decompress the second compressed data using a compression engine to obtain the second initial data, the main device can obtain the second initial data from the expansion device through a single data copy. Compared to existing technologies, only one data copy is required between the main device and the expansion device, reducing the interaction between the main device and the expansion device, thereby improving the efficiency of data acquisition.

[0196] Figure 8 This is a structural diagram of a data processing device provided in an embodiment of the present application. Figure 8 The data processing device 10 can be applied to an expansion device of a main device. The data processing device 10 may include: a receiving module 11, a processing module 12 and a storage module 13, wherein:

[0197] The receiving module 11 is configured to receive a data storage request sent by the master device, wherein the data storage request includes first initial data;

[0198] The processing module 12 is configured to compress the first initial data using the compression engine to obtain first compressed data;

[0199] The storage module 13 is configured to store the first compressed data in the extended memory.

[0200] In a possible implementation, the expansion device further includes a temporary cache, and the data storage request is used to request writing the first initial data into the temporary cache; and the processing module 12 is specifically configured to:

[0201] When the master device finishes writing the first initial data into the temporary buffer, obtaining the first initial data from the temporary buffer by the compression engine;

[0202] The first initial data is compressed to obtain the first compressed data.

[0203] In a possible implementation, multiple virtual address spaces are provided in the expansion device, wherein the first initial data is written by the main device into the temporary cache through a first virtual address space, and the first virtual address space is an idle virtual address space among the multiple virtual address spaces.

[0204] The data processing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment. Its implementation principles and beneficial effects are similar and will not be repeated here.

[0205] Figure 9 This is a structural diagram of another data processing device provided in an embodiment of the present application. Figure 9 ,exist Figure 8 Based on the embodiment shown, the data processing device 10 may further include a determination module 14, wherein:

[0206] The determining module 14 is configured to determine a first actual storage space of the first compressed data in the extended memory;

[0207] The determining module 14 is further configured to determine a mapping relationship between the first actual storage space and the first virtual address space;

[0208] The storage module 13 is further configured to store the mapping relationship.

[0209] The data processing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment. Its implementation principles and beneficial effects are similar and will not be repeated here.

[0210] In a possible implementation, the data processing device 10 further includes: a sending module 15,

[0211] The sending module 15 is configured to send a data storage response to the master device after acquiring the first initial data in the temporary buffer through the compression engine;

[0212] The data storage response is used to indicate completion of the first initial data storage, and the data storage response is used to enable the main device to asynchronously store other data to the expansion device.

[0213] In a possible implementation, after the compression engine obtains the first initial data in the temporary cache, the apparatus further includes a deleting module 16:

[0214] The deleting module 16 is configured to delete the first initial data in the temporary cache.

[0215] In a possible implementation, the device further includes an acquisition module 17 and a synchronization module 18, wherein:

[0216] The receiving module 11 is further configured to receive a data read request sent by the master device;

[0217] The acquisition module 17 is configured to acquire the second compressed data from the extended memory according to the data read request;

[0218] The processing module 12 is configured to decompress the second compressed data through the compression engine to obtain second initial data;

[0219] The synchronization module 18 is further configured to synchronize the second initial data to the master device.

[0220] In a possible implementation, the data read request includes an identifier of the second virtual address space; and the acquisition module 17 is specifically configured to:

[0221] determining a second actual storage space according to the identifier of the second virtual address space;

[0222] The second compressed data is obtained in the extended memory according to the second actual storage space.

[0223] In a possible implementation, the synchronization module 18 is specifically configured to:

[0224] storing the second initial data in a temporary buffer of the expansion device;

[0225] A data read response is sent to the master device, where the data read response is used to instruct the master device to read the second initial data in the temporary cache through the second virtual address space.

[0226] The data processing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment. Its implementation principles and beneficial effects are similar and will not be repeated here.

[0227] The exemplary embodiment of the present application provides a structural diagram of an expansion device, see Figure 10 The expansion device 20 may include a processor 21, a memory 22, a compression engine 23, and an expansion memory 24. Exemplarily, the processor 21, the memory 22, the compression engine 23, and the expansion memory 24 are interconnected via a bus 25.

[0228] The memory 22 stores computer-executable instructions;

[0229] The compression engine 23 is used to compress and / or decompress data;

[0230] The extended memory 24 is used to store data;

[0231] The processor 21 executes the computer-executable instructions stored in the memory 22 , so that the expansion device 20 executes the method shown in the above method embodiment.

[0232] Accordingly, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the above method embodiment.

[0233] Accordingly, an embodiment of the present application may also provide a computer program product, including a computer program, which, when executed by a processor, may implement the method shown in the above method embodiment.

[0234] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0235] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0236] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0237] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0238] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0239] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0240] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0241] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0242] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: An expansion device applied to a main device, the expansion device including a compression engine and an expansion memory, the method comprising: receiving a data storage request sent by the master device, wherein the data storage request includes first initial data; compressing the first initial data by the compression engine to obtain first compressed data; The first compressed data is stored in the extended memory.

2. The method according to claim 1, characterized in that The expansion device further includes a temporary cache, and the data storage request is used to request writing the first initial data into the temporary cache; Compressing the first initial data by the compression engine to obtain first compressed data includes: When the master device finishes writing the first initial data into the temporary buffer, obtaining the first initial data from the temporary buffer by the compression engine; The first initial data is compressed to obtain the first compressed data.

3. The method according to claim 2, characterized in that The expansion device is provided with a plurality of virtual address spaces, wherein the first initial data is written by the main device into the temporary cache through a first virtual address space, and the first virtual address space is an idle virtual address space among the plurality of virtual address spaces.

4. The method according to claim 3, characterized in that After storing the first compressed data in the extended memory, the method further includes: determining a first actual storage space of the first compressed data in the extended memory; A mapping relationship between the first actual storage space and the first virtual address space is determined, and the mapping relationship is stored.

5. The method according to any one of claims 2 to 4, characterized in that: The method further comprises: After acquiring the first initial data in the temporary buffer by the compression engine, sending a data storage response to the master device; The data storage response is used to indicate completion of the first initial data storage, and the data storage response is used to enable the main device to asynchronously store other data to the expansion device.

6. The method according to any one of claims 2 to 5, characterized in that: After obtaining the first initial data from the temporary cache through the compression engine, the method further includes: The first initial data is deleted from the temporary buffer.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: receiving a data read request sent by the master device; acquiring second compressed data in the extended memory according to the data read request; The second compressed data is decompressed by the compression engine to obtain second initial data, and the second initial data is synchronized to the master device.

8. The method according to claim 7, characterized in that The data read request includes an identifier of the second virtual address space; and obtaining the second compressed data in the extended memory according to the data read request includes: determining a second actual storage space according to the identifier of the second virtual address space; The second compressed data is obtained in the extended memory according to the second actual storage space.

9. The method according to claim 8, characterized in that Synchronizing the second initial data to the master device includes: storing the second initial data in a temporary buffer of the expansion device; A data read response is sent to the master device, where the data read response is used to instruct the master device to read the second initial data in the temporary cache through the second virtual address space.

10. A data processing device, characterized in that: An expansion device applied to a main device, the expansion device includes a compression engine and an expansion memory, the device includes: a receiving module, a processing module and a storage module, wherein: The receiving module is configured to receive a data storage request sent by the master device, wherein the data storage request includes first initial data; The processing module is configured to compress the first initial data through the compression engine to obtain first compressed data; The storage module is used to store the first compressed data in the extended memory.

11. An expansion device, characterized in that: include: at least one processor; as well as a memory, a compression engine, and an expansion memory communicatively coupled to the at least one processor; The compression engine is used to compress and / or decompress data; The extended memory is used to store data; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the expansion device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images