Memory performance optimization method for database, electronic equipment and storage medium

Abstract

The invention provides a memory performance optimization method for a database, electronic equipment and a storage medium, and is applied to the technical field of databases. The method comprises the following steps of: mounting a storage-level memory as a data memory heap, initializing the storage-level memory and setting the storage-level memory as a memory mode; reading data of a certain table for the first time, loading data blocks from a disk to a data memory stack, marking the data blocks as hot data, and converting some data blocks into cold data after the hot data memory is full according to an LIRS mechanism; and starting an asynchronous thread, and monitoring cold data of the data memory heap. And releasing the signal, reading the cold data by the thread, and asynchronously writing the cold data into the storage-level memory. And when the storage-level memory is full of the data blocks, cleaning the cold data blocks according to an LIRS mechanism, and landing the cold data blocks to a disk. The data of a certain table is read again, the data memory heap is searched firstly, then the storage-level memory is searched, and if the data cache exists, the data can be directly used. And if the loading from the disk is not carried out again, circularly executing the operations of first reading, thread monitoring and re-reading.

Description

Technical Field

[0001] The present invention relates to the technical field of database and memory optimization, and in particular to a memory performance optimization method, electronic device and storage medium for a database. Background Art

[0002] A database is a warehouse for storing and managing data, and memory plays an important role in database management. Because memory speeds up the reading and writing of data, the response time is greatly reduced, allowing users to quickly obtain the required data. Usually, data that is frequently read in the database is cached in memory, thereby turning multiple write operations of these frequently read data into one write operation, reducing the dependence on disk IO, and greatly improving database performance. However, when the memory is too small, data will be frequently swapped in and out, and some data will be released from the memory and fall back to the disk, which will lead to reduced performance and slower data processing. In addition, compared with disks, memory is more expensive and has a smaller capacity. If you blindly pursue database performance and deploy too much memory in hardware, it will cost a lot of money, resulting in a very low cost performance. Summary of the invention

[0003] In view of the above problems, the present invention provides a memory optimization method for a database, an electronic device and a storage medium, in order to solve at least one of the above problems.

[0004] According to a first aspect of the present invention, there is provided a method for optimizing memory performance of a database, comprising:

[0005] Mount the storage-level memory as the data memory heap. When starting a stand-alone service through the database cluster, call the corresponding interface of the storage-level memory to complete the initialization of the storage-level memory, and set the working mode of the storage-level memory to memory mode.

[0006] When the target data table is read for the first time, the target data block in the target data table is loaded from the disk into the data memory heap and the target data block is marked as hot data. Based on the predefined page replacement mechanism, if the target data block exceeds the hot data memory threshold, part of the data in the target data block is converted to cold data.

[0007] The cold data in the data memory heap is monitored by starting an asynchronous thread, and based on the obtained signal, the cold data in the data memory heap is read through the asynchronous thread and the cold data in the data memory heap is asynchronously written into the storage-level memory;

[0008] When the storage-level memory is full of data blocks, the cold data in the storage-level memory is cleaned up according to the predefined page replacement mechanism, and the cleaned up cold data is stored in the disk;

[0009] When the target data table is read again, the data cache conditions of the data memory heap and the storage-level memory are retrieved in sequence, and based on the retrieval results, the target data blocks in the target data table are read or reloaded from the disk to the data memory heap for processing.

[0010] According to an embodiment of the present invention, the above-mentioned predefined page replacement mechanism includes a LIRS mechanism, and the storage-level memory includes an AEP.

[0011] According to an embodiment of the present invention, when the target data table is read for the first time, the target data block in the target data table is loaded from the disk into the data memory heap and the target data block is marked as hot data. Based on a predefined page replacement mechanism, when the target data block exceeds the hot data memory threshold, part of the data in the target data block is converted to cold data, including:

[0012] A node information table is created for each target data block loaded into the data memory heap to record the information of the target data block;

[0013] The node information table of the previous target data block is written into the node table, and then the node information table of the current target data block is written into the node table and the node information table of the previous target data block is deleted, wherein the node table is a collection of node information tables and is updated through the new node table;

[0014] In the process of loading the target data block in the target data table from the disk to the data memory heap, the node information table creation operation and the node table update operation are repeatedly executed until the data in the target data table is cached;

[0015] When the data in the target data table is completely cached, data blocks exceeding the hot data memory threshold are marked as cold data according to a predefined page replacement mechanism, and a temporary node information table for recording the data blocks marked as cold data is generated.

[0016] According to an embodiment of the present invention, the node information table includes a table unique identifier, a column unique identifier, a data block unique identifier, and a data block pointer object.

[0017] According to an embodiment of the present invention, the above-mentioned monitoring of cold data in the data memory heap by starting an asynchronous thread, reading the cold data in the data memory heap through the asynchronous thread based on the obtained signal and asynchronously writing the cold data in the data memory heap into the storage-level memory includes:

[0018] After some data marked as cold data in the target data block sends a signal, the cold data is read through the database's asynchronous thread;

[0019] The information of cold data is obtained by reading the temporary node information table through an asynchronous thread, and the corresponding cold data is found based on the obtained information of the cold data, and the found cold data is written into the storage-level memory.

[0020] According to an embodiment of the present invention, the above-mentioned monitoring of cold data in the data memory heap by starting an asynchronous thread, reading the cold data in the data memory heap through the asynchronous thread based on the obtained signal and asynchronously writing the cold data in the data memory heap to the storage-level memory also includes:

[0021] Memory is freed by clearing out the cold data already in the data memory heap, and a node information table identical to that of the target data block is created for the cold data written to the storage-level memory.

[0022] According to an embodiment of the present invention, when the target data table is read again, the data cache conditions of the data memory heap and the storage-level memory are retrieved successively, and based on the retrieval results, the target data block in the target data table is read or reloaded from the disk into the data memory heap for processing, including:

[0023] First search the data memory heap and then search the storage-level memory to obtain the search result;

[0024] When the retrieval result is that the data cache of the target data table exists in the data memory heap or the storage-level memory, the data cache in the data memory heap or the storage-level memory is directly read based on the node information table of the target data block in the target data table.

[0025] According to an embodiment of the present invention, when the target data table is read again, the data cache conditions of the data memory heap and the storage-level memory are retrieved successively, and based on the retrieval result, the target data block in the target data table is read or reloaded from the disk into the data memory heap for processing, and further includes:

[0026] If the retrieval result is that the data cache of the target data table does not exist in the data memory heap and the storage-level memory, the loading operation of the target data block from the disk to the data memory heap, the reading and writing operations of the asynchronous thread, and the cleaning operation of the cold data are re-executed.

[0027] According to a second aspect of the present invention, there is provided an electronic device, comprising:

[0028] one or more processors;

[0029] a storage device for storing one or more programs,

[0030] When one or more programs are executed by one or more processors, the one or more processors execute a memory performance optimization method for a database.

[0031] According to a third aspect of the present invention, there is provided a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute a memory performance optimization method for a database.

[0032] The memory performance optimization method for a database provided by the present invention uses storage-level memory as a secondary cache, which solves the problem of frequently swapping in and out of commonly used data in the database due to small memory, effectively improves data performance, and facilitates user operations on the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of a method for optimizing memory performance of a database according to an embodiment of the present invention;

[0034] Figure 2 A block diagram of an electronic device suitable for implementing a memory performance optimization method for a database according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0036] This invention provides a method of using storage-level memory (such as AEP: Apache Pass Memory, Intel Optane persistent memory) as a secondary cache of memory to expand memory, solving the problem that frequently used data is frequently swapped in and out due to small memory, thus affecting database performance. AEP has the advantages of low price, large capacity, and read and write speeds similar to memory. Using its memory storage mode to expand memory can better solve the above technical problems.

[0037] Figure 1 The present invention is a flowchart of a method for optimizing the memory performance of a database according to an embodiment of the present invention.

[0038] like Figure 1 As shown, the above-mentioned memory performance optimization method for a database includes operations S110 to S150.

[0039] Operation S110, mount the storage-level memory as a data memory heap, call the corresponding interface of the storage-level memory when starting a stand-alone service through the database cluster to complete the initialization of the storage-level memory, and set the working mode of the storage-level memory to the memory mode.

[0040] The above-mentioned hot data represents data with high access frequency; the above-mentioned cold data represents data with low access frequency; the above-mentioned data memory heap is used to cache commonly used data and temporary data, that is, the DC heap; the above-mentioned storage-level memory, optionally, includes AEP, that is, Intel Optane Data Center Persistent Memory, which has 3 working modes: storage mode, memory mode and hybrid mode.

[0041] Before performing storage-level memory initialization, you can set the memory size of hot and cold data according to user needs, that is, the threshold of cold data and the threshold of hot data. For example, when using the gbase database, you can set the size of commonly used hot data according to your needs, or directly use the default option of gbase.

[0042] If AEP is used as the storage-level memory and gbase is selected as the database, the above operation S110 can be expressed as: AEP is mounted as a DC heap, and when the gbase database cluster starts the gbased service, the interface corresponding to AEP is called to initialize it, and the working mode of AEP is set to memory mode, and the memory size of hot and cold data is defined.

[0043] Operation S120, when the target data table is read for the first time, the target data block in the target data table is loaded from the disk into the data memory heap and the target data block is marked as hot data. Based on a predefined page replacement mechanism, when the target data block exceeds the hot data memory threshold, part of the data in the target data block is converted to cold data.

[0044] According to an embodiment of the present invention, the predefined page replacement mechanism includes a LIRS (Low Inter-reference Recency Set: a low inter-reference recent set, which belongs to a page replacement algorithm) mechanism, and the storage-class memory includes an AEP.

[0045] If AEP is used as the storage-level memory and gbase is selected as the database, the above operation S120 can be described as: when a table data is read for the first time, the data dc is loaded from the disk to the DC heap and marked as hot data. According to the LIRS mechanism, when the hot data memory is full, some data dc is converted to cold data.

[0046] Operation S130, monitoring the cold data in the data memory heap by starting an asynchronous thread, reading the cold data in the data memory heap through the asynchronous thread based on the obtained signal, and asynchronously writing the cold data in the data memory heap into the storage-level memory.

[0047] Start an asynchronous thread to monitor the cold data in the DC heap. Release the signal, the thread reads the cold data and writes it asynchronously to the AEP.

[0048] Operation S140, when the storage-level memory is full of data blocks, the cold data in the storage-level memory is cleared according to a predefined page replacement mechanism, and the cleared cold data is stored in the disk.

[0049] When the data DC in AEP is full, the cold data DC is cleaned up according to the LIRS mechanism and landed on the disk.

[0050] Operation S150, when reading the target data table again, the data cache conditions of the data memory heap and the storage-level memory are searched successively, and based on the search result, the target data block in the target data table is read or reloaded from the disk to the data memory heap for processing.

[0051] When reading the data of a table again, first search the DC heap, then search the AEP, and if the data is cached, use it directly. If not, load it from the disk again, and execute S120~S140 in a loop.

[0052] The memory performance optimization method for a database provided by the present invention uses storage-level memory as a secondary cache, which solves the problem of frequently swapping in and out of commonly used data in the database due to small memory, effectively improves data performance, and facilitates user operations on the database.

[0053] According to an embodiment of the present invention, when the target data table is read for the first time, the target data block in the target data table is loaded from the disk into the data memory heap and the target data block is marked as hot data. Based on a predefined page replacement mechanism, when the target data block exceeds the hot data memory threshold, part of the data in the target data block is converted into cold data, including: creating a node information table for each target data block loaded into the data memory heap to record the information of the target data block; writing the node information table of the previous target data block into the node table, then writing the node information table of the current target data block into the node table and deleting the node information table of the previous target data block, wherein the node table is a collection of node information tables and is updated through a new node table; in the process of loading the target data block in the target data table from the disk into the data memory heap, repeatedly performing the node information table creation operation and the node table update operation until the data in the target data table is cached; when the data in the target data table is cached, according to the predefined page replacement mechanism, the data block exceeding the hot data memory threshold is marked as cold data, and a temporary node information table for recording the data block marked as cold data is generated.

[0054] According to an embodiment of the present invention, the node information table includes a table unique identifier, a column unique identifier, a data block unique identifier, and a data block pointer object.

[0055] The following is a specific example in which AEP is used as the storage-level memory and gbase is used as the database to further illustrate the process of reading the target data table for the first time.

[0056] In operation S120, a table_list table for recording data dc information is generated, and the specific operations are as follows: each time data dc is loaded into the DC stack, a new table_list table for recording DC information is created; first, the table_list information of the data dc last loaded into the DC stack is written into the new table, and then the data dc information is written into the new table_list; the table_list table created by the last data dc is deleted, and the above two steps are repeated, namely "each time data dc is loaded into the DC stack, a new table_list table for recording DC information is created" and "first, the table_list information of the data dc last loaded into the DC stack is written into the new table, and then the data dc information is written into the new table_list", until the data dc is cached; according to the lirs mechanism, the data dc is marked as cold data; at the same time, a temporary table_list table (tmp_list) for recording cold data dc is generated; further, the created table_list table information includes table id, column id, data dc_id and data dc pointer object.

[0057] According to an embodiment of the present invention, the above-mentioned monitoring of cold data in the data memory heap by starting an asynchronous thread, reading the cold data in the data memory heap through the asynchronous thread based on the obtained signal and asynchronously writing the cold data in the data memory heap into the storage-level memory includes: after part of the data marked as cold data in the target data block sends a signal, starting to read the cold data through the asynchronous thread of the database; reading the temporary node information table through the asynchronous thread to obtain the information of the cold data, and finding the corresponding cold data based on the obtained information of the cold data, and writing the found cold data into the storage-level memory; freeing up the memory by clearing the existing cold data in the data memory heap, and creating a node information table identical to the target data block for the cold data written into the storage-level memory.

[0058] The following is a specific example, using AEP as the storage-level memory and gbase as the database used, to further illustrate the related operation process of the above asynchronous thread.

[0059] Operation S130 is mainly used to detect the generated cold data: when cold data exists, a signal is sent, a background thread starts to read the cold data, the thread reads the cold data dc information in tmp_list, finds the corresponding data, and writes it into AEP; the DC pile cold data is cleared to release the memory;

[0060] According to an embodiment of the present invention, when the target data table is read again, the data cache conditions of the data memory heap and the storage-level memory are retrieved successively, and based on the retrieval result, the target data block in the target data table is read or reloaded from the disk to the data memory heap for processing, including: first searching the data memory heap and then searching the storage-level memory to obtain the retrieval result; when the retrieval result is that the data cache of the target data table exists in the data memory heap or the storage-level memory, based on the node information table of the target data block in the target data table, directly reading the data cache in the data memory heap or the storage-level memory; when the retrieval result is that the data cache of the target data table does not exist in both the data memory heap and the storage-level memory, re-executing the loading operation of the target data block from the disk to the data memory heap, the reading and writing operations of the asynchronous thread, and the cleaning operation of the cold data.

[0061] The following is a specific example in which AEP is used as the storage-level memory and gbase is used as the database to further illustrate the above-mentioned process of re-reading the target data table.

[0062] The above operation S150 searches for data dc according to table_list in memory: when the DC heap and AEP have data cache, when reading a certain table data, first read the DC heap table_list, then read the table_list in AEP; first read the table id in table_list, then the column id and data dc_id; if the data is found in the memory cache, read it directly. If the corresponding information is not found, it indicates that the data has been stored in the disk, and the data should be loaded from the disk to the data memory.

[0063] The above-mentioned method for optimizing the memory performance of a database provided by the present invention effectively improves the hit rate of data in the memory cache. When a large amount of data is read into the memory, the commonly used cached data will not be marked as cold data and then released to the disk due to the small memory size, but will be asynchronously written to the AEP. When reading this cold data again, it does not need to be loaded from the disk, but directly read from the AEP, thereby improving the read and write rate of data in the database. This reduces the frequent swapping of data between the memory and the disk, which results in a long reading time and affects performance.

[0064] The memory optimization method for a database provided by the present invention is further described in detail below through another embodiment.

[0065] First, install AEP on the server and other hardware, then restart the cluster gbased service. The program calls the AEP related interface and initializes the AEP device, using its memory mode.

[0066] It should be noted that the switch parameter is added to the code program. Users log in to the cluster and set this parameter on and off to control whether to cache data in the AEP. At the same time, when initializing the AEP device, the cold and hot memory sizes of the AEP are defined according to the corresponding ratio.

[0067] After AEP is initialized, data can be cached, which is mainly divided into three parts:

[0068] Part 1: Memory DC heap cache. When a user logs in to the cluster and performs the first data query on a table, the data is cached from the disk to the memory DC heap. The specific process is as follows:

[0069] (11) The user logs in to the cluster and selects an existing library table for query.

[0070] (12) The database cluster sends instructions to each node, selects the correct data DC, caches it in the DC heap of the memory, and generates a table_list containing the data DC information.

[0071] (13) First, mark the data DC as hot data and use the LIRS mechanism. When the hot data memory is full, it is converted into cold data.

[0072] Part 2: Cold data monitoring. When executing step (12), the background monitoring thread enters a waiting state. When hot data is transferred to cold data, a signal is issued to read the cold data and write it to AEP. When transferring cold data, a temporary table_list that records the cold data DC information is synchronously generated. If the cold data heap is empty or there is no available DC, it is in a waiting state.

[0073] It should be noted that the memory DC heap not only caches data DC, but also caches temporary table data DC. However, temporary table data DC is not allowed to be transferred to AEP. It is cleared after the task is completed to release the DC heap memory. Only cold data DC can trigger the background thread, and temporary table data DC cannot be triggered.

[0074] Part 3: AEP cache. When cold data is written to AEP, it is first marked as hot data according to the lirs mechanism. When the hot data memory is full, it is converted to cold data. When the cold data memory is full, the cold data is cleaned and landed on the disk.

[0075] After the data is read for the first time and cached in the memory and AEP, the response time is significantly shortened when the data is read again. The specific process of reading data is as follows: the user queries the data of a table multiple times, and the cluster sends instructions to the single machine; each node reads the DC heap and table_list in AEP in turn, and finds the corresponding data DC information based on the conditions; if not found, it is determined that the data has been written to the disk; and the data is read from the disk again.

[0076] At the same time, a memory monitoring module is provided to help users understand memory usage. The details are as follows: Users log in to a single-machine cluster and query the memory table to monitor the single-machine DC heap and AEP memory usage. However, the query results are real-time and can only represent the current memory usage. The table format is shown in Table 1:

[0077] Table 1

[0078]

[0079] Figure 2 A block diagram of an electronic device suitable for implementing a memory performance optimization method for a database according to an embodiment of the present invention is schematically shown.

[0080] like Figure 2 As shown, the electronic device 200 according to an embodiment of the present invention includes a processor 201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 202 or the program loaded from the storage part 208 to the random access memory (RAM) 203. The processor 201 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 201 may also include an onboard memory for caching purposes. The processor 201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0081] In RAM 203, various programs and data required for the operation of electronic device 200 are stored. Processor 201, ROM 202 and RAM 203 are connected to each other via bus 204. Processor 201 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 202 and / or RAM 203. It should be noted that the program can also be stored in one or more memories other than ROM 202 and RAM 203. Processor 201 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0082] According to an embodiment of the present invention, the electronic device 200 may further include an input / output (I / O) interface 205, which is also connected to the bus 204. The electronic device 200 may further include one or more of the following components connected to the I / O interface 205: an input portion 206 including a keyboard, a mouse, etc.; an output portion 207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 208 including a hard disk, etc.; and a communication portion 209 including a network interface card such as a LAN card, a modem, etc. The communication portion 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to the I / O interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed, so that a computer program read therefrom is installed into the storage portion 208 as needed.

[0083] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0084] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 202 and / or RAM 203 described above and / or one or more memories other than ROM 202 and RAM 203.

[0085] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0086] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for optimizing memory performance of a database, characterized in that: include: Mount the storage-level memory as a data memory heap, call the corresponding interface of the storage-level memory to complete the initialization of the storage-level memory when starting a stand-alone service through the database cluster, and set the working mode of the storage-level memory to the memory mode; When a target data table is read for the first time, a target data block in the target data table is loaded from a disk into the data memory heap and the target data block is marked as hot data, and based on a predefined page replacement mechanism, when the target data block exceeds a hot data memory threshold, part of the data in the target data block is converted to cold data; The cold data in the data memory heap is monitored by starting an asynchronous thread, and based on the obtained signal, the cold data in the data memory heap is read by the asynchronous thread and the cold data in the data memory heap is asynchronously written into the storage-level memory; When the storage-level memory is full of data blocks, according to the predefined page replacement mechanism, the cold data in the storage-level memory is cleaned up, and the cleaned up cold data is stored in the disk; When the target data table is read again, the data cache conditions of the data memory heap and the storage-level memory are searched successively, and based on the search results, the target data blocks in the target data table are read or reloaded from the disk to the data memory heap for processing.

2. The method according to claim 1, characterized in that The predefined page replacement mechanism includes a LIRS mechanism, and the storage-class memory includes an AEP.

3. The method according to claim 1, characterized in that When the target data table is read for the first time, a target data block in the target data table is loaded from the disk into the data memory heap and the target data block is marked as hot data, and based on a predefined page replacement mechanism, when the target data block exceeds a hot data memory threshold, part of the data in the target data block is converted to cold data, including: Creating a node information table for each target data block loaded into the data memory heap to record information of the target data block; Writing the node information table of the previous target data block into the node table, then writing the node information table of the current target data block into the node table and deleting the node information table of the previous target data block, wherein the node table is a collection of the node information tables and is updated through a new node table; In the process of loading the target data block in the target data table from the disk to the data memory heap, repeatedly performing the creation operation of the node information table and the update operation of the node table until the data in the target data table is completely cached; When the data in the target data table is completely cached, the data blocks exceeding the hot data memory threshold are marked as cold data according to a predefined page replacement mechanism, and a temporary node information table for recording the data blocks marked as cold data is generated.

4. The method according to claim 3, characterized in that: The node information table includes a table unique identifier, a column unique identifier, a data block unique identifier, and a data block pointer object.

5. The method according to claim 3, characterized in that: Monitoring the cold data in the data memory heap by starting an asynchronous thread, reading the cold data in the data memory heap by the asynchronous thread based on the obtained signal, and asynchronously writing the cold data in the data memory heap into the storage-level memory includes: After a portion of the data marked as cold data in the target data block sends a signal, the cold data is started to be read through an asynchronous thread of the database; The information of the cold data is obtained by reading the temporary node information table through the asynchronous thread, and the corresponding cold data is found based on the obtained information of the cold data, and the found cold data is written into the storage-level memory.

6. The method according to claim 5, characterized in that Also includes: Memory is released by clearing out cold data already existing in the data memory heap, and a node information table identical to the target data block is created for the cold data written into the storage-level memory.

7. The method according to claim 1, characterized in that When reading the target data table again, searching the data cache conditions of the data memory heap and the storage-level memory in sequence, and reading the target data block in the target data table or reloading it from the disk to the data memory heap for processing based on the search result includes: Firstly search the data memory heap and then search the storage level memory to obtain a search result; In the case where the retrieval result is that the data cache of the target data table exists in the data memory heap or the storage-level memory, the data cache in the data memory heap or the storage-level memory is directly read based on the node information table of the target data block in the target data table.

8. The method according to claim 7, characterized in that Also includes: When the retrieval result is that the data cache of the target data table does not exist in the data memory heap and the storage-level memory, the loading operation of the target data block from the disk to the data memory heap, the reading and writing operations of the asynchronous threads, and the cleaning operation of the cold data are re-executed.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 8.

Images