Data processing method, data transfer method, data processing apparatus, computing device, computer readable storage medium and computer program product

Abstract

The embodiment of the invention provides a data processing method and device, the data processing method is applied to a computing node, the computing node comprises a second-level cache hosted in a target storage medium, and the data processing method comprises the following steps: determining an initial data page meeting a data rejection condition in a memory; updating the initial data page to a target data page, and transferring the target data page to the second-level cache based on a data transmission channel between the memory and the second-level cache; and under the condition that a data reading request associated with the target data page is received, loading the target data page in the second-level cache to the memory for executing a data processing task corresponding to the data reading request. When the data in the target data page needs to be read, the target data page stored in the second-level cache is loaded to the memory, so that the data reading efficiency is improved, and the data processing performance of the computing node is improved.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a data processing method, a data transfer method, a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the continuous development of information technology, people need to save more and more information and data, and the demand for storage capacity is increasing. In addition, as software and hardware upgrades, various applications have higher and higher performance requirements for storage devices. When accessing data, the relevant data pages must first be loaded from persistent storage into memory, and then data is read and written on the data pages loaded into memory. Since the size of the memory is limited, after the memory is full, if you want to access data in a data page that is in persistent storage but not in memory, you need to remove another data page from the memory to reserve storage space for the data page to be accessed. Due to the low efficiency of reading and writing data in persistent storage, the efficiency of reading and writing data in memory is ultimately low, and the data processing performance of the storage device is low. Therefore, a more effective data processing method is urgently needed to solve this problem. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a data transfer method, a computing device, a computer-readable storage medium, and a computer program product to address technical deficiencies in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a data processing method is provided, which is applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, comprising:

[0005] Determine the initial data page in memory that meets the data elimination conditions;

[0006] Updating the initial data page to a target data page, and transferring the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache;

[0007] When a data read request associated with the target data page is received, the target data page in the secondary cache is loaded into the memory for executing a data processing task corresponding to the data read request.

[0008] According to a second aspect of an embodiment of this specification, a data transfer method is provided, which is applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, including:

[0009] Determine the initial data page in memory that meets the data elimination conditions;

[0010] Updating the initial data page to a target data page, and transferring the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache;

[0011] The target data page stored in the secondary cache is used to load into the memory and then perform data processing tasks.

[0012] According to a third aspect of an embodiment of this specification, a data processing device is provided, which is applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, including:

[0013] a determination module configured to determine an initial data page in the memory that meets a data elimination condition;

[0014] a transfer module configured to update the initial data page to a target data page, and transfer the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache;

[0015] The loading module is configured to load the target data page in the secondary cache into the memory when receiving a data read request associated with the target data page, so as to execute a data processing task corresponding to the data read request.

[0016] According to a fourth aspect of an embodiment of this specification, a data transfer device is provided, which is applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, including:

[0017] a determination module configured to determine an initial data page in the memory that meets a data elimination condition;

[0018] The transfer module is configured to update the initial data page to a target data page, and transfer the target data page to the secondary cache based on the data transmission channel between the memory and the secondary cache; wherein the target data page stored in the secondary cache is used to execute data processing tasks after being loaded into the memory.

[0019] According to a fifth aspect of the embodiments of this specification, there is provided a computing device, including:

[0020] memory and processor;

[0021] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented.

[0022] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned data processing method are implemented.

[0023] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0024] The data processing method provided by one embodiment of the present specification is applied to a computing node, and the computing node includes a secondary cache hosted on a target storage medium, including: determining an initial data page in the memory that meets the data elimination condition; updating the initial data page to a target data page, and transferring the target data page to the secondary cache based on the data transmission channel between the memory and the secondary cache; when a data read request for the associated target data page is received, the target data page in the secondary cache is loaded into the memory for executing the data processing task corresponding to the data read request. The secondary cache hosted on the target storage medium in the computing node is used to store the initial data page eliminated from the memory, and the initial data page is updated to the target data page and stored in the secondary cache, thereby realizing temporary storage for the initial data page that cannot be stored in the memory. Therefore, when the data in the target data page needs to be read, the target data page stored in the secondary cache is loaded into the memory, thereby improving the data reading efficiency and further improving the data processing performance of the computing node. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of a processing process of a data processing method provided by an embodiment of this specification;

[0026] Figure 2 is a flow chart of a data processing method provided by one embodiment of this specification;

[0027] Figure 3 This is a flowchart of a data processing method provided by one embodiment of this specification;

[0028] Figure 4 This is a schematic diagram of the structure of a computing node in a data processing method provided by one embodiment of this specification;

[0029] Figure 5 This is a schematic diagram of the structure of a data processing device provided by one embodiment of this specification;

[0030] Figure 6 This is a flow chart of a data transfer method provided by one embodiment of this specification;

[0031] Figure 7This is a structural diagram of a data transfer device provided by an embodiment of this specification;

[0032] Figure 8 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0033] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0034] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0035] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0036] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0037] First, the terms involved in one or more embodiments of this specification are explained.

[0038] LRU (Least Recently Used) is a common cache eviction algorithm. Its core idea is to evict the least recently used data when the cache is full. The LRU algorithm maintains the cache by recording the access time of data items. When a new data item needs to be inserted, if the cache is full, the least recently used data item is evicted.

[0039] LRU-2: A variation of the LRU algorithm, it implements two cache queues. When data is first accessed, it is cached in a FIFO (first-in, first-out) queue. When data is accessed a second time, it is moved from the FIFO queue to the LRU queue. Each queue eliminates data in its own way.

[0040] BP (Buffer Pool): The buffer pool is a memory cache in a relational database system. Data pages on the storage are accessed through this memory cache.

[0041] L2C (Level-2 Cache): A secondary cache located between the BP and storage. L2C is hosted in local persistent memory, local SSD, or remote memory. L2C data pages are compressed.

[0042] Data pages: In storage, data is physically organized into fixed-size pages, which are the access units between the relational database system and storage.

[0043] Data blocks: A group or several groups of records arranged sequentially. They are the basic unit of data storage and the smallest physical storage unit.

[0044] Relational database system: A system that provides storage, access, and modification capabilities for data, in which data is logically organized in tables.

[0045] Transparent compression: A data compression technology that is transparent to users and applications during its use. Users and applications do not need to perform any actions or be aware of the data compression process. Transparent compression is commonly used in storage and network transmission scenarios, effectively reducing data storage space and transmission time.

[0046] DRAM: Dynamic Random Access Memory, a common type of computer memory, uses capacitors to store data and maintains it through a constantly refreshing circuit. DRAM is characterized by its high speed, high integration, and low power consumption, making it widely used in computer systems as both main memory and cache memory.

[0047] Figure 1 This is a schematic diagram of a data processing method provided by an embodiment of this specification; Figure 1As shown, the data processing method is applied to a computing node corresponding to a relational database system, and the computing node includes a secondary cache hosted on a target storage medium. When there is a data page that needs to be eliminated in the memory, an initial data page in the memory that meets the data elimination conditions is determined. The initial data page is compressed into a target data page, and the target data page is transferred to the secondary cache based on the data transmission channel between the memory and the secondary cache. When a data read request associated with the target data page is received, the target data page in the secondary cache is loaded into the memory for executing the data processing task corresponding to the data read request.

[0048] The compute node utilizes the secondary cache hosted on the target storage medium to store the initial data page removed from memory. The initial data page is then updated to the target data page and stored in the secondary cache, temporarily storing the initial data page that cannot be stored in memory. When the data in the target data page needs to be read, the target data page stored in the secondary cache is loaded into memory, improving data reading efficiency and, in turn, the compute node's data processing performance.

[0049] In this specification, a data processing method is provided. This specification also relates to a data processing device, a data transfer method, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0050] See also Figure 2 , Figure 2 A flow chart of a data processing method provided according to an embodiment of the present specification is shown. The data processing method is applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, and specifically includes the following steps.

[0051] Step 202: Determine the initial data page in the memory that meets the data elimination condition.

[0052] Specifically, the memory is an internal memory used to temporarily store calculation data and data exchanged with external memory such as a hard disk; the data elimination condition is used to determine whether a data page in the memory needs to be eliminated from the memory; the data elimination condition may correspond to the version information of the data page in the memory, the update timestamp of the data page, or the data access frequency of the data page in the memory; accordingly, the initial data page is the data page in the memory that meets the data elimination condition.

[0053] Based on this, the data elimination conditions corresponding to the memory are determined, the data pages stored in the memory are detected based on the data elimination conditions, the initial data page is determined according to the detection result, and the initial data page is eliminated from the memory.

[0054] In practical applications, when determining the initial data page in memory, the data page to be eliminated from the memory can be determined based on the LRU algorithm or the LRU-2 algorithm, eliminating the least recently used data page in the memory.

[0055] Furthermore, considering that data pages in the memory need to be stored in the secondary cache after being eliminated, in order not to affect the data pages already stored in the secondary cache when the data pages eliminated from the memory are written to the secondary cache, the target data pages can be stored in the secondary cache in an appended manner. In addition, since the storage space occupied by data pages is generally small, in order to improve the storage efficiency and storage standardization of data pages, the target data pages can be first stored in the secondary cache area in the secondary cache. When the secondary cache area is full, all data pages stored in the secondary cache area are transferred to the secondary cache segment in the secondary cache in the form of data blocks. The specific implementation is as follows:

[0056] A secondary cache area is determined in the secondary cache, and the target data page is additionally stored in the secondary cache area based on a data transmission channel between the memory and the secondary cache; when it is determined that the secondary cache area meets the data transfer condition based on the storage result of the additional storage of the target data page, at least one data page contained in the secondary cache area is integrated into a data block, and the data block is transferred to the secondary cache segment in the secondary cache.

[0057] Specifically, the secondary cache area is a storage space in the secondary cache used to temporarily store data pages removed from the memory; the secondary cache area can store data pages written in real time, and can also store garbage data pages, and release the space occupied by garbage data pages; the storage result refers to the storage status information after the target data page is stored in the secondary cache area, and the storage result can indicate that the target data page has been stored in the secondary cache area; the data transfer condition can correspond to the number of data pages stored in the secondary cache area, and the storage space occupied by the data page; the data block is the data block obtained by integrating all the data pages contained in the secondary cache area; the secondary cache segment is a storage space with a larger storage unit in the secondary cache, and its storage unit is a data block, while the storage unit of the secondary cache area is a data page; the data blocks in the secondary cache segment are written in an appended manner.

[0058] Based on this, the L2 cache consists of L2 cache areas and L2 cache segments. L2 cache areas are used to store data pages from memory, while L2 cache segments are used to store data blocks from L2 cache areas. Within the L2 cache, a L2 cache area is identified, and the target data page is appended to the L2 cache area based on the data transfer channel between memory and the L2 cache. Whether the secondary cache area meets the data transfer condition is judged according to the storage result of the additional storage of the target data page. When it is judged according to the storage result of the additional storage of the target data page that the secondary cache area meets the data transfer condition, it means that the number of data pages stored in the secondary cache area has reached the storage number threshold, or the storage space occupied by the data pages stored in the secondary cache area has reached the storage space threshold. At this time, at least one data page contained in the secondary cache area is integrated into a data block, and the data block is transferred to the secondary cache segment in the secondary cache; on the contrary, when it is judged according to the storage result of the additional storage of the target data page that the secondary cache area does not meet the data transfer condition, it means that the number of data pages stored in the secondary cache area has not reached the storage number threshold, or the storage space occupied by the data pages stored in the secondary cache area has not reached the storage space threshold. At this time, there is no need to perform data transfer operation on the secondary cache area, and you can wait for the next data page to be written to the secondary cache area.

[0059] For example, the size of a data page in memory is fixed at 16KB. After the data page in memory is removed, it is stored in the L2 cache area of ​​the L2 cache. The storage threshold in the L2 cache area is 2MB. When the data page stored in the L2 cache area reaches 2MB, the data pages corresponding to these 2MB are integrated into a data block, and the data block is transferred to the L2 cache segment in the L2 cache. It should be noted that the storage threshold in the L2 cache area can be set according to actual needs and is not limited in this embodiment.

[0060] In summary, if the L2 cache area determines that the data transfer conditions are met based on the storage result of the target data page's appended storage, at least one data page contained in the L2 cache area is integrated into a data block, and the data block is transferred to a L2 cache segment within the L2 cache. This improves the storage efficiency and storage standardization of the data pages, and improves data write efficiency by appending data to the target data page.

[0061] Furthermore, considering that the data pages removed from the memory need to be transferred to the secondary cache, and data read and write operations may be performed on the data pages removed from the memory later, in order to ensure the read and write accuracy of the data pages removed from the memory, the mapping information can be determined after the data pages are transferred to the secondary cache, and the mapping information can be stored in the data page mapping table. The specific implementation is as follows:

[0062] Determine the storage location information of the target data page, wherein the storage location information is used to indicate the storage location of the target data page in the secondary cache; generate mapping information based on the storage location information and the page identifier of the target data page, and store the mapping information in a data page mapping table, wherein the mapping information stored in the data page mapping table is used to determine the data page to be read in the secondary cache.

[0063] Specifically, the storage location information indicates the storage location of the target data page in the secondary cache; the page identifier can be a mark such as the data page ID, attribute information, etc. that can identify the target data page; the mapping information includes the storage location information and page identifier of the target data page. When the page identifier of the data page is determined, the storage location corresponding to the data page can be determined based on the mapping information; accordingly, the data page mapping table is used to store the mapping information of all data pages in the computing node, and is used to accurately find the data pages that need to be read and written when reading and writing data.

[0064] Based on this, the storage location of the target data page in the L2 cache is determined. Mapping information is generated based on the target data page's storage location in the L2 cache and the target data page's page identifier to record the target data page's storage location in the L2 cache. The mapping information is stored in a data page mapping table. The mapping information stored in the data page mapping table is used to determine the data page to be read in the L2 cache when a read request to the L2 cache is received.

[0065] Continuing with the above example, after the target data page is stored in the secondary cache, mapping information is generated based on the storage location information of the target data page in the secondary cache and the page identifier of the target data page, and the mapping information is stored in the secondary cache so that the target data page can be accurately found when data read and write operations are performed on the target data page subsequently.

[0066] In summary, mapping information is generated based on the storage location information and the page identifier of the target data page, and the mapping information is stored in the data page mapping table, so that the target data page can be accurately found during subsequent read and write operations on the target data page, thereby improving data operation efficiency.

[0067] Furthermore, considering that the secondary cache continuously stores data pages that are removed from the memory, and the data pages stored in the secondary cache may be changed into garbage data pages over time, these garbage data pages will occupy a large amount of storage space in the secondary cache. Therefore, it is necessary to release the storage space occupied by the garbage data pages in the secondary cache. The specific implementation is as follows:

[0068] Performing garbage data detection on the secondary cache, determining garbage data pages according to the detection result; and deleting the garbage data pages from the secondary cache.

[0069] Specifically, the garbage data detection operation is used to detect garbage data pages contained in the secondary cache. The garbage data pages may be data pages in the secondary cache that are no longer needed, marked as deleted, or have become invalid.

[0070] Based on this, before storing the target data page in the L2 cache, the L2 cache is checked for garbage data, and based on the check results, the garbage data pages contained in the L2 cache are determined, and the garbage data pages are deleted from the L2 cache. In actual applications, the L2 cache can also be checked for garbage data at fixed intervals to determine the garbage data pages contained in the L2 cache, and the garbage data pages are deleted to free up the storage space occupied by the garbage data pages in the L2 cache.

[0071] Continuing with the previous example, before storing the target data page in the L2 cache, we can perform a garbage page check on the L2 cache and delete the garbage pages in the L2 cache to free up storage space for the target data page. We can also periodically perform garbage page checks on the L2 cache to reduce the proportion of garbage pages in the L2 cache.

[0072] In summary, performing garbage data detection on the secondary cache and deleting garbage data pages in the secondary cache can free up storage space for storing target data pages and improve the availability of the secondary cache.

[0073] Step 204: Update the initial data page to a target data page, and transfer the target data page to the secondary cache based on the data transmission channel between the memory and the secondary cache.

[0074] Specifically, after the initial data page that meets the data elimination conditions in the memory is determined as mentioned above, the initial data page can be updated to the target data page, and based on the data transmission channel between the memory and the secondary cache, the target data page can be transferred to the secondary cache, wherein the target data page refers to the data page obtained after the initial data page is updated. The update can be performed on the initial data page to compress and encode it, thereby reducing the storage space occupied by the initial data page, or the data structure of the initial data page can be changed to update the data structure of the initial data page to a data structure that matches the secondary cache and complies with the storage structure of the secondary cache; the data transmission channel is used to provide support for data transmission between the memory and the secondary cache. The data transmission channel can be a physical channel or a logical channel, and can be a medium that supports data transmission, such as a network connection, a communication protocol, etc.

[0075] Based on this, after determining the initial data page in the memory that meets the data culling criteria, the initial data page is updated and the target data page is determined by compressing the initial data page. A data transmission channel between the memory and the secondary cache is determined, and the target data page is transferred to the secondary cache based on the data transmission channel.

[0076] In practical applications, both memory and L2 cache serve as storage media within compute nodes. While memory offers higher data read and write efficiency, L2 cache offers lower data read and write efficiency. L2 cache resides between memory and storage (persistent storage). It is hosted on local persistent memory, local SSDs, or remote memory. The process of compressing initial data pages that have been evicted from memory into target data pages and storing them in L2 cache is a dynamic data processing process. Data pages that have been evicted from memory can be stored in L2 cache in real time.

[0077] Furthermore, considering that the data reading and writing speed of the secondary cache is lower than that of the memory, and the data reading and writing speed of the storage medium is higher than that of the persistent storage space, the storage space of the secondary cache is limited. In order to increase the number of data pages that can be stored in the secondary cache, the initial data page can be compressed before being stored in the secondary cache, and then the compressed initial data page is stored in the secondary cache; in addition, since the memory and the secondary cache are two different storage media, the data structure corresponding to the memory and the data structure corresponding to the secondary cache may also be different. Therefore, it is necessary to modify the storage structure of the initial data page before storing it in the secondary cache. The specific implementation is as follows:

[0078] The initial data page is compressed to obtain a target data page; or the initial data page of the initial data structure is converted into the target data page of the target data structure matching the secondary cache.

[0079] Specifically, compression refers to the change in the representation and organization of the target data page at the physical storage level. During the data page compression process, the logical structure and relational structure of the data remain unchanged, reducing the storage space occupied by the target data. The compression of the target data page can be achieved by transparent compression. The initial data structure is the storage form of the initial data page in the memory. Correspondingly, the target data structure is the storage form of the target data page in the secondary cache.

[0080] Based on this, the initial data page is compressed using a transparent compression method to obtain the target data page, and the initial data page is updated to the target data page; or, the initial data page of the initial data structure is converted into the target data page of the target data structure that matches the secondary cache, and the initial data page is updated to the target data page.

[0081] Continuing with the previous example, if the initial data page removed from memory is 16KB, the 16KB initial data page is transparently compressed to reduce the storage space occupied by the data page, and the initial data page is updated through compression. Because memory and L2 cache are two different storage media, the data structure corresponding to memory and L2 cache may also differ. Therefore, before storing the target data page in the L2 cache, the storage structure of the target data page must be modified to match the storage structure of the L2 cache. The initial data page is updated by modifying the data structure.

[0082] In summary, the initial data page is updated by compressing it, thereby improving the utilization of the secondary cache; the initial data page is updated by changing the initial data structure corresponding to the initial data page, thereby improving the data matching degree between the target data page and the secondary cache.

[0083] Furthermore, considering the limited storage space in the secondary cache, it is necessary to clean up the data pages in the secondary cache to ensure that there is enough storage space in the secondary cache to store the data pages that are removed from the memory. When deleting data pages in the secondary cache, it is necessary to determine whether the data page needs to be deleted directly or stored in the persistent storage space to avoid deleting valid data. The specific implementation is as follows:

[0084] When it is determined that the target data page meets the data deletion condition corresponding to the secondary cache, determine whether the version information of the target data page meets the version deletion condition; if so, delete the target data page in the secondary cache; if not, transfer the target data page to the storage space contained in the computing node, and delete the target data page in the secondary cache.

[0085] Specifically, the data deletion conditions may be the storage time length, storage order, and data page version; for example, the storage time of the data page in the secondary cache exceeds a preset storage time threshold; the version of the data page stored in the secondary cache is too low, or has not been updated for a long time; satisfying the version deletion condition indicates that the data page version stored in the secondary cache of the target data page is low, and also indicates that the target data page has not been updated or read or written for a long time in the secondary cache; the storage space contained in the computing node may be a persistent storage space.

[0086] Based on this, determine whether the target data page meets the data deletion conditions corresponding to the secondary cache. If it is determined that the target data page meets the data deletion conditions corresponding to the secondary cache, it means that the target data page needs to be deleted due to version factors or storage time factors. At this time, determine whether the version information of the target data page meets the version deletion conditions; if the version information of the target data page meets the version deletion conditions, it means that the version of the target data page is relatively low and the target data page has not been updated for a long time. The target data page can be deleted from the secondary cache; if the version information of the target data page does not meet the version deletion conditions, it means that the version of the target data page is relatively high and the target data page has not been updated for a short time. The target data page can be transferred to the storage space contained in the computing node and the target data page can be deleted from the secondary cache.

[0087] Continuing with the above example, if the target data page is a new version data page, the target data page needs to be stored in the persistent storage space for preservation, and then the target data page in the secondary cache is deleted; if the target data page is not a new version data page, the target data page in the secondary cache can be directly deleted.

[0088] To summarize, before discarding the target data page in the secondary cache, determine whether the target data page meets the version deletion condition, and then decide whether to directly delete the target data page in the secondary cache or store the target data page in the persistent storage space for permanent storage, so as to ensure that the new version of the target data page will not be deleted.

[0089] Step 206: When a data read request associated with the target data page is received, the target data page in the secondary cache is loaded into the memory for executing a data processing task corresponding to the data read request.

[0090] Specifically, after the initial data page is updated to the target data page and the target data page is transferred to the secondary cache based on the data transmission channel between the memory and the secondary cache, when a data read request associated with the target data page is received, the target data page in the secondary cache can be loaded into the memory to execute the data processing task corresponding to the data read request, wherein the data read request refers to a computer instruction for obtaining the data stored in the target data page in the secondary cache; the data processing task can be a data task executed based on the data in the target data page, including but not limited to data sorting, data analysis, data sharing, and machine learning tasks.

[0091] Based on this, after the initial data page is updated to the target data page and the target data page is transferred to the secondary cache based on the data transmission channel between the memory and the secondary cache, when a data read request associated with the target data page is received, it indicates that the data to be read corresponding to the data read request is stored in the target data page in the secondary cache, and the target data page in the secondary cache is loaded into the memory based on the data read request for continuing to execute the data processing task corresponding to the data read request.

[0092] Furthermore, considering that there are a large number of data pages stored in the secondary cache, in order to improve retrieval efficiency, the target data page can be determined in the secondary cache based on the data page mapping table. The specific implementation is as follows:

[0093] Determine a data read identifier based on the data read request; determine data location information corresponding to the data read identifier in the data page mapping table; determine the target data page in the secondary cache based on the data location information, and load the target data page into the memory.

[0094] Specifically, the data read identifier represents the attribute information of the data that needs to be read corresponding to the data read request. The attribute information may be the ID of the data page, the name of the data page, and other information; the data location information is the location information corresponding to the data read identifier recorded in the data page mapping table. The data location information corresponding to the data read identifier can be queried according to the data page mapping table; the data location information represents the storage location of the data page in the secondary cache.

[0095] Based on this, the data read request is parsed to obtain the data read identifier carried in the data read request. The data location information corresponding to the data read identifier is determined in the data page mapping table. Based on the data location information, the target data page is determined in the secondary cache. The target data page is read and loaded into the memory.

[0096] Continuing with the previous example, the data page mapping table records the data read identifier for each data page and the data location information stored on each data page. The data read identifier and data location information are stored in the data page mapping table as a single record. When searching for a target data page, the data read identifier is used to determine the target data page's storage location in the data page mapping table, and the target data page can then be read from that storage location.

[0097] In summary, the target data page corresponding to the data read request is read from the secondary cache based on the data page mapping table, thereby improving the accuracy of target data page search and improving data reading efficiency.

[0098] Furthermore, considering that when a data acquisition request is submitted to the memory, the storage location of the data page corresponding to the data acquisition request may be the memory or the secondary cache, when a data acquisition request is received, it is necessary to determine whether the data acquisition request is associated with the memory or the secondary cache, and then perform targeted data page reading. The specific implementation is as follows:

[0099] A data acquisition request for the memory is received, and the data acquisition request is parsed to obtain a data page identifier; when it is determined based on the data page mapping table that the data page identifier is associated with the memory, the target data corresponding to the data acquisition request is read in the memory based on the data page mapping table, so as to execute the data task corresponding to the data acquisition request; when it is determined based on the data page mapping table that the data page identifier is associated with the secondary cache, the data acquisition request is used as the data read request, and the step of determining the data read identifier based on the data read request is executed.

[0100] Specifically, the data acquisition request is used to obtain data from the computing node; the data page identifier represents the attribute information such as the data page ID to be obtained, and the data page identifier is used to mark and identify the data page; the target data is the data that needs to be read corresponding to the data acquisition request, and the data task is the data operation based on the target data, such as data analysis, data sharing, etc.

[0101] Based on this, a data acquisition request for the memory is received. The data acquisition request is parsed to obtain a data page identifier. In the case where the data page identifier is determined to be associated with the memory based on the data page mapping table, it means that the data page corresponding to the data page identifier is stored in the memory, and the target data corresponding to the data acquisition request can be read in the memory based on the data page mapping table to execute the data task corresponding to the data acquisition request. In the case where the data page identifier is determined to be associated with the secondary cache based on the data page mapping table, it means that the data page corresponding to the data page identifier is stored in the secondary cache, and the data acquisition request can be used as a data read request, and the data read identifier is determined based on the data read request, and then the data location information corresponding to the data read identifier is determined in the data page mapping table. The target data page is determined in the secondary cache based on the data location information. The target data page is read, and the target data page is loaded into the memory, and then the target data is determined in the target data page loaded into the memory.

[0102] In summary, when a data acquisition request is received, it is determined whether the data page identifier corresponding to the data acquisition request is associated with the memory or the secondary cache, and then the target data is read in a targeted manner, thereby improving the accuracy of data reading.

[0103] The data processing method provided by one embodiment of the present specification is applied to a computing node, and the computing node includes a secondary cache hosted on a target storage medium, including: determining an initial data page in the memory that meets the data elimination condition; updating the initial data page to a target data page, and transferring the target data page to the secondary cache based on the data transmission channel between the memory and the secondary cache; when a data read request for the associated target data page is received, the target data page in the secondary cache is loaded into the memory for executing the data processing task corresponding to the data read request. The secondary cache hosted on the target storage medium in the computing node is used to store the initial data page eliminated from the memory, and the initial data page is updated to the target data page and stored in the secondary cache, thereby realizing temporary storage for the initial data page that cannot be stored in the memory. Therefore, when the data in the target data page needs to be read, the target data page stored in the secondary cache is loaded into the memory, thereby improving the data reading efficiency and further improving the data processing performance of the computing node.

[0104] The following combined Figure 3 , taking the application of the data processing method provided in this specification to read data in a relational database system as an example, the data processing method is further explained. Figure 3 A flowchart of a data processing method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0105] Step 302: Determine the initial data pages in the buffer pool that meet the data elimination conditions.

[0106] In a relational database system, data is organized into fixed-size data pages and stored persistently in memory. To access data, the relevant data pages must first be loaded into a memory cache, known as the buffer pool. The buffer pool has a finite size. If the buffer pool is full, accessing data from a page that is in storage but not in the buffer pool requires evicting another page from the buffer pool to make room for the desired page. Whether data eviction conditions are met can be determined using a LRU or LRU-2 algorithm.

[0107] Step 304: compress the initial data page to obtain a target data page.

[0108] After determining the initial data page eliminated from the buffer pool, the initial data page is compressed using a transparent compression method to obtain the target data page.

[0109] Step 306 : Determine a secondary cache area in the secondary cache, and append the target data page to the secondary cache area based on the data transmission channel between the buffer pool and the secondary cache.

[0110] Step 308: When it is determined that the L2 cache area meets the data transfer condition, at least one data page contained in the L2 cache area is integrated into a data block, and the data block is transferred to the L2 cache segment in the L2 cache.

[0111] If the L2 cache meets the data transfer conditions, it means that the L2 cache is full and has reached the preset storage threshold. The storage threshold can be set to any value such as 2M based on actual needs.

[0112] Step 310: Determine the storage location information of the target data page in the L2 cache.

[0113] Step 312: Generate mapping information based on the storage location information and the page identifier of the target data page, and store the mapping information in a page mapping table.

[0114] The target data page's storage location in the L2 cache is used to locate the target data page during read and write operations. The page identifier can be a data page ID, which maps the data page ID to the data page's storage location. The page mapping table can be managed by DRAM (dynamic random access memory).

[0115] Step 314 : When a data read request associated with a target data page is received, determine a data read identifier based on the data read request.

[0116] Step 316: Determine data location information corresponding to the data read identifier in the page mapping table.

[0117] Step 318: Determine the target data page in the secondary cache based on the data location information, and load the target data page into the buffer pool for executing the data processing task corresponding to the data read request.

[0118] When a buffer pool user needs to read a data page, if the data page exists in the buffer pool, the data is read from the buffer pool and fed back to the user. If the data page is not in the buffer pool, the second-level cache is checked. If the page mapping table determines that the data page is in the second-level cache, the data page in the second-level cache is decompressed and loaded into the buffer pool. After the data page is loaded from the second-level cache to the buffer pool, it can be discarded from the second-level cache.

[0119] In addition, if the data page that the buffer pool user needs to read does not exist in the secondary cache, the data page can be directly determined in the storage space and loaded into the buffer pool. The secondary cache can also be checked for garbage data, and based on the detection results, the garbage data page can be determined and deleted from the secondary cache, or the data page that is still valid can be copied forward. In the case where the target data page is deleted from the secondary cache, it is determined whether the version information of the target data page meets the version deletion condition; if it does, it means that the version of the target data page is lower, and the target data page is deleted from the secondary cache; if it does not meet the condition, it means that the version of the target data page is higher, and the target data page is transferred to the storage space contained in the computing node, and the target data page is deleted from the secondary cache.

[0120] In practical applications, such as Figure 4 As shown, a compute node includes a buffer pool, a L2 cache, and storage space. The L2 cache area in the L2 cache has real-time write and garbage collection capabilities. After data pages in the buffer pool are removed, they can be written to the L2 cache area in the L2 cache. When the L2 cache area is full, the data pages are consolidated into data blocks and transferred to the L2 cache segment for storage. Data pages in the L2 cache can also be written to the storage space for persistent storage. Similarly, data pages in the buffer pool can also be written to the storage space for persistent storage. When reading data, if the data page is stored in the buffer pool, the data page can be directly read from the buffer pool. If the data page is stored in the L2 cache, the actual storage location of the data page can be determined based on the page mapping table, and the data page can then be loaded into the buffer pool for data reading. If the data page is stored in the storage space, the data page in the storage space can be loaded into the buffer pool for data reading.

[0121] In summary, the initial data pages are compressed before being stored in the secondary cache, thereby increasing cache density, enabling more data pages to be stored in the secondary cache, and improving data page access performance. As an extension of the buffer pool, the secondary cache has greater deployment flexibility. The hosting media of the secondary cache can be of different types, such as local permanent storage, local SSD, and remote storage. They share the same architecture. When the secondary cache segments are located across the network, because the data pages are appended to the secondary cache area in compressed form, this helps reduce the demand for network bandwidth. In the secondary cache, real-time writes and garbage collection use different segments. Data pages appended by real-time writes are more likely to be changed (i.e., have a higher temperature) than data pages appended by garbage collection. This allows for the initial separation of pages of different temperatures, which helps reduce write amplification caused by garbage collection.

[0122] The data processing method provided by one embodiment of the present specification is applied to a computing node, and the computing node includes a secondary cache hosted on a target storage medium, including: determining an initial data page in the buffer pool that meets the data removal condition; updating the initial data page to a target data page, and transferring the target data page to the secondary cache based on the data transmission channel between the buffer pool and the secondary cache; when a data read request for the associated target data page is received, the target data page in the secondary cache is loaded into the buffer pool for executing the data processing task corresponding to the data read request. The secondary cache hosted on the target storage medium in the computing node is used to store the initial data page removed from the buffer pool, and the initial data page is updated to the target data page and stored in the secondary cache, thereby realizing temporary storage for the initial data page that cannot be stored in the buffer pool. Therefore, when the data in the target data page needs to be read, the target data page stored in the secondary cache is loaded into the buffer pool, thereby improving the data reading efficiency and further improving the data processing performance of the computing node.

[0123] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 5 FIG1 shows a schematic diagram of the structure of a data processing device provided by an embodiment of this specification. Figure 5 As shown, the device is applied to a computing node, and the computing node includes a secondary cache hosted on a target storage medium, including:

[0124] A determination module 502 is configured to determine an initial data page in the memory that meets a data elimination condition;

[0125] A transfer module 504 is configured to update the initial data page to a target data page, and transfer the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache;

[0126] The loading module 506 is configured to, upon receiving a data read request associated with the target data page, load the target data page in the secondary cache into the memory for executing a data processing task corresponding to the data read request.

[0127] In an optional embodiment, the determining module 502 is further configured to:

[0128] Determine a secondary cache area in the secondary cache, and append the target data page to the secondary cache area based on a data transmission channel between the memory and the secondary cache;

[0129] When it is determined that the secondary cache area meets the data transfer condition based on the storage result of the additional storage of the target data page, at least one data page contained in the secondary cache area is integrated into a data block, and the data block is transferred to the secondary cache segment in the secondary cache.

[0130] In an optional embodiment, the determining module 502 is further configured to:

[0131] Determining storage location information of the target data page, wherein the storage location information is used to indicate a storage location of the target data page in the secondary cache;

[0132] Mapping information is generated based on the storage location information and the page identifier of the target data page, and the mapping information is stored in a data page mapping table. The mapping information stored in the data page mapping table is used to determine the data page to be read in the secondary cache.

[0133] In an optional embodiment, the determining module 502 is further configured to:

[0134] compressing the initial data page to obtain a target data page; or,

[0135] An initial data page of an initial data structure is converted into the target data page of a target data structure matching the secondary cache.

[0136] In an optional embodiment, the determining module 502 is further configured to:

[0137] Performing garbage data detection on the secondary cache, and determining garbage data pages according to the detection result;

[0138] The garbage data page is deleted from the secondary cache.

[0139] In an optional embodiment, the transfer module 504 is further configured to:

[0140] If it is determined that the target data page meets the data deletion condition corresponding to the secondary cache, determining whether the version information of the target data page meets the version deletion condition;

[0141] If so, deleting the target data page from the secondary cache;

[0142] If not, the target data page is transferred to the storage space included in the computing node, and the target data page is deleted from the secondary cache.

[0143] In an optional embodiment, the loading module 506 is further configured to:

[0144] determining a data read identifier based on the data read request;

[0145] Determining data location information corresponding to the data read identifier in the data page mapping table;

[0146] The target data page is determined in the secondary cache based on the data location information, and the target data page is loaded into the memory.

[0147] In an optional embodiment, the loading module 506 is further configured to:

[0148] receiving a data acquisition request for the memory, and parsing the data acquisition request to obtain a data page identifier;

[0149] In a case where it is determined based on the data page mapping table that the data page identifier is associated with the memory, reading the target data corresponding to the data acquisition request from the memory based on the data page mapping table, so as to execute the data task corresponding to the data acquisition request;

[0150] In the case where it is determined based on the data page mapping table that the data page identifier is associated with the secondary cache, the data acquisition request is used as the data read request, and the step of determining the data read identifier based on the data read request is performed.

[0151] The data processing device provided by one embodiment of the present specification is applied to a computing node, and the computing node includes a secondary cache hosted on a target storage medium, including: determining an initial data page in the memory that meets the data elimination condition; updating the initial data page to a target data page, and transferring the target data page to the secondary cache based on the data transmission channel between the memory and the secondary cache; when a data read request for the associated target data page is received, the target data page in the secondary cache is loaded into the memory for executing the data processing task corresponding to the data read request. The secondary cache hosted on the target storage medium in the computing node is used to store the initial data page eliminated from the memory, and the initial data page is updated to the target data page and stored in the secondary cache, thereby realizing temporary storage for the initial data page that cannot be stored in the memory. Therefore, when the data in the target data page needs to be read, the target data page stored in the secondary cache is loaded into the memory, thereby improving the data reading efficiency and further improving the data processing performance of the computing node.

[0152] The above is a schematic diagram of a data processing device according to this embodiment. It should be noted that the technical solution of the data processing device and the technical solution of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical solution of the data processing device, please refer to the description of the technical solution of the above-mentioned data processing method.

[0153] See also Figure 6 , Figure 6 A flow chart of a data transfer method provided according to an embodiment of the present specification is shown. The data transfer method is applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, and specifically includes the following steps.

[0154] Step 602: Determine the initial data page in the memory that meets the data elimination condition;

[0155] Step 604: Update the initial data page to a target data page, and transfer the target data page to the secondary cache based on the data transmission channel between the memory and the secondary cache; wherein the target data page stored in the secondary cache is used to execute data processing tasks after being loaded into the memory.

[0156] In practical applications, after determining the initial data page in memory that meets the data culling criteria, the initial data page is updated and compressed to determine the target data page. A data transmission channel between the memory and the L2 cache is determined, and the target data page is transferred to the L2 cache based on the data transmission channel. The initial data page is updated to the target data page and stored in the L2 cache, temporarily storing the initial data page that cannot be stored in memory. Therefore, when data in the target data page needs to be read, the target data page stored in the L2 cache is loaded into the memory, improving data reading efficiency and, in turn, the data processing performance of the compute node.

[0157] Corresponding to the above method embodiment, this specification also provides a data transfer device embodiment, Figure 7 FIG1 shows a schematic diagram of the structure of a data transfer device provided by an embodiment of this specification. Figure 7 As shown, the device is applied to a computing node, and the computing node includes a secondary cache hosted on a target storage medium, including:

[0158] A determination module 702 is configured to determine an initial data page in the memory that meets a data elimination condition;

[0159] The transfer module 704 is configured to update the initial data page to the target data page, and transfer the target data page to the secondary cache based on the data transmission channel between the memory and the secondary cache; wherein the target data page stored in the secondary cache is used to execute data processing tasks after being loaded into the memory.

[0160] In an optional embodiment, the determining module 702 is further configured to:

[0161] Determine a secondary cache area in the secondary cache, and append the target data page to the secondary cache area based on a data transmission channel between the memory and the secondary cache;

[0162] When it is determined that the secondary cache area meets the data transfer condition based on the storage result of the additional storage of the target data page, at least one data page contained in the secondary cache area is integrated into a data block, and the data block is transferred to the secondary cache segment in the secondary cache.

[0163] In an optional embodiment, the determining module 702 is further configured to:

[0164] Determining storage location information of the target data page, wherein the storage location information is used to indicate a storage location of the target data page in the secondary cache;

[0165] Mapping information is generated based on the storage location information and the page identifier of the target data page, and the mapping information is stored in a data page mapping table. The mapping information stored in the data page mapping table is used to determine the data page to be read in the secondary cache.

[0166] In an optional embodiment, the determining module 702 is further configured to:

[0167] compressing the initial data page to obtain a target data page; or,

[0168] An initial data page of an initial data structure is converted into the target data page of a target data structure matching the secondary cache.

[0169] In an optional embodiment, the determining module 702 is further configured to:

[0170] Performing garbage data detection on the secondary cache, and determining garbage data pages according to the detection result;

[0171] The garbage data page is deleted from the secondary cache.

[0172] In an optional embodiment, the transfer module 704 is further configured to:

[0173] If it is determined that the target data page meets the data deletion condition corresponding to the secondary cache, determining whether the version information of the target data page meets the version deletion condition;

[0174] If so, deleting the target data page from the secondary cache;

[0175] If not, the target data page is transferred to the storage space included in the computing node, and the target data page is deleted from the secondary cache.

[0176] The above is a schematic diagram of a data transfer device according to this embodiment. It should be noted that the technical solution of the data transfer device and the technical solution of the above-mentioned data transfer method are based on the same concept. For details not described in detail in the technical solution of the data transfer device, please refer to the description of the technical solution of the above-mentioned data transfer method.

[0177] Figure 8 8 shows a block diagram of a computing device 800 according to one embodiment of the present disclosure. Components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0178] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0179] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 8 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0180] Computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 800 may also be a mobile or stationary server.

[0181] The processor 820 is configured to execute the following computer-executable instructions, which implement the steps of the above method when executed by the processor.

[0182] The above is a schematic solution of a computing device of this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above method belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above method.

[0183] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which implement the steps of the above method when executed by a processor.

[0184] The above is a schematic solution of a computer-readable storage medium of this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the above method belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above method.

[0185] An embodiment of the present specification further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0186] The above is an illustrative solution of a computer program product of this embodiment. It should be noted that the technical solution of the computer program product and the technical solution of the above method belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above method.

[0187] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0188] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0189] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0190] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0191] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, comprising: Determine the initial data page in memory that meets the data elimination conditions; Updating the initial data page to a target data page, and transferring the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache; When a data read request associated with the target data page is received, the target data page in the secondary cache is loaded into the memory for executing a data processing task corresponding to the data read request.

2. The data processing method according to claim 1, wherein transferring the target data page to the secondary cache based on the data transmission channel between the memory and the secondary cache comprises: Determine a secondary cache area in the secondary cache, and append the target data page to the secondary cache area based on a data transmission channel between the memory and the secondary cache; When it is determined that the secondary cache area meets the data transfer condition based on the storage result of the additional storage of the target data page, at least one data page contained in the secondary cache area is integrated into a data block, and the data block is transferred to the secondary cache segment in the secondary cache.

3. The data processing method according to claim 1, further comprising: Determining storage location information of the target data page, wherein the storage location information is used to indicate a storage location of the target data page in the secondary cache; Mapping information is generated based on the storage location information and the page identifier of the target data page, and the mapping information is stored in a data page mapping table. The mapping information stored in the data page mapping table is used to determine the data page to be read in the secondary cache.

4. The data processing method according to claim 1, wherein updating the initial data page to the target data page comprises: Compressing the initial data page to obtain a target data page; or, An initial data page of an initial data structure is converted into the target data page of a target data structure matching the secondary cache.

5. The data processing method according to claim 1 , wherein before the initial data page is updated to a target data page and the target data page is transferred to the secondary cache based on a data transmission channel between the memory and the secondary cache, the method further comprises: Performing garbage data detection on the secondary cache, and determining garbage data pages according to the detection result; The garbage data page is deleted from the secondary cache.

6. The data processing method according to claim 1 , further comprising: updating the initial data page to a target data page and transferring the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache; If it is determined that the target data page meets the data deletion condition corresponding to the secondary cache, determining whether the version information of the target data page meets the version deletion condition; If so, deleting the target data page from the secondary cache; If not, the target data page is transferred to the storage space included in the computing node, and the target data page is deleted from the secondary cache.

7. The data processing method according to claim 3, wherein loading the target data page in the secondary cache into the memory comprises: determining a data read identifier based on the data read request; Determining data location information corresponding to the data read identifier in the data page mapping table; The target data page is determined in the secondary cache based on the data location information, and the target data page is loaded into the memory.

8. The data processing method according to claim 7, further comprising: updating the initial data page to a target data page and transferring the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache; receiving a data acquisition request for the memory, and parsing the data acquisition request to obtain a data page identifier; In a case where it is determined based on the data page mapping table that the data page identifier is associated with the memory, reading the target data corresponding to the data acquisition request from the memory based on the data page mapping table, so as to execute the data task corresponding to the data acquisition request; In the case where it is determined based on the data page mapping table that the data page identifier is associated with the secondary cache, the data acquisition request is used as the data read request, and the step of determining the data read identifier based on the data read request is performed.

9. A data transfer method, applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, comprising: Determine the initial data page in memory that meets the data elimination conditions; Updating the initial data page to a target data page, and transferring the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache; The target data page stored in the secondary cache is used to load into the memory and then perform data processing tasks.

10. A data processing device, applied to a computing node, wherein the computing node includes a secondary cache hosted on a target storage medium, comprising: a determination module configured to determine an initial data page in the memory that meets a data elimination condition; a transfer module configured to update the initial data page to a target data page, and transfer the target data page to the secondary cache based on a data transmission channel between the memory and the secondary cache; The loading module is configured to load the target data page in the secondary cache into the memory when receiving a data read request associated with the target data page, so as to execute a data processing task corresponding to the data read request.

11. A computing device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program or instructions, which implement the steps of the method according to any one of claims 1 to 9 when executed by a processor.

Images