Method for accelerating database hotspot data and electronic device

Abstract

The present invention provides a method for accelerating hotspot data in a database and an electronic device, which are applied to the fields of computer technology and database technology. The acceleration method includes: in response to detecting a data write operation, determining the incremental data of the target row according to the pre-write data and post-write data of the target row targeted by the data write operation; when it is determined that the registered data of the target row already exists in the row buffer pool, writing the incremental data into the row version chain corresponding to the target row in the incremental buffer pool; in response to determining that the target row is hotspot data, when the complete row data of the target row does not exist in the row buffer pool, writing the post-write data as the complete row data into the row buffer pool, so as to calculate the version data of the target row to be read according to the row version chain and the complete row data in response to a read request. Therefore, the present invention can at least solve the technical problem of obvious access delay and achieve the technical effect of accelerating hotspot data and reducing access delay.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and database technology, and more specifically, to a method for accelerating database hotspot data and an electronic device. Background Art

[0002] The data of traditional non-memory databases are all stored on disk. To relieve the pressure of disk input / output (I / O), the buffer pool is introduced as the memory component of the database to cache the real data on the disk. The addition, deletion, modification and query actions performed by the application system on the database are mainly performed on the data in the memory data structure. Usually, the basic unit of data exchange between memory and disk is data page, referred to as page, so the buffer pool stores multiple pages.

[0003] When a user needs to read a row of data, the page where the row of data is located will be loaded into the buffer pool, and the row of data required by the user will be filtered out and returned. Ideally, if the buffer pool is set large enough, it is equivalent to storing the entire database in memory, and the user's read operation does not need to access the disk. However, the size of the buffer pool of an actual database is limited. Although the buffer pool's least recently used (LRU) rule elimination and pre-reading can keep the data in the pages that users are about to access or frequently access in the buffer pool as much as possible, reducing the access to the disk during reading, access delays are still inevitable in actual use, affecting the user experience of users who focus on latency.

[0004] Therefore, the prior art urgently needs an acceleration method for reducing access delay. Summary of the invention

[0005] In view of this, the present invention provides a method for accelerating hotspot data in a database and an electronic device.

[0006] One aspect of the present invention provides a method for accelerating hot spot data in a database, comprising: in response to detecting a data write operation, determining the incremental data of the target row according to the pre-write data and post-write data of the target row targeted by the data write operation; when it is determined that the registered data of the target row already exists in the row buffer pool, writing the incremental data into the row version chain corresponding to the target row in the incremental buffer pool, wherein the row buffer pool and the incremental buffer pool are both set in the memory; in response to determining that the target row is hot spot data, when the complete row data of the target row does not exist in the row buffer pool, writing the post-write data into the row buffer pool as the complete row data, so as to calculate the version data of the target row to be read according to the row version chain and the complete row data in response to a read request.

[0007] Optionally, the acceleration method also includes: when writing the incremental data into the row version chain corresponding to the target row, synchronously writing the target row's belonging row information, index information and data visible range into the row version chain.

[0008] Optionally, the row version chain includes the incremental data of at least one data write operation; the acceleration method also includes: determining the validity of the incremental data of each data write operation according to the data visibility range; deleting the invalid incremental data in the row version chain of the incremental buffer pool; and deleting the complete row data from the row buffer pool when the incremental data of each data write operation in the row version chain are invalid.

[0009] Optionally, determining that the target row is hot data is achieved in the following manner: obtaining a reading frequency of the target row in a first preset time period; and determining that the target row is hot data when the reading frequency is greater than a frequency threshold.

[0010] Optionally, the acceleration method also includes: when it is determined that the target row is hot data, establishing a daemon thread for the target row to call the daemon thread to calculate the heat value of the target row, and when the heat value continues to be lower than the heat threshold, modifying the hot attribute for the target row in the row buffer pool to non-hot data.

[0011] Optionally, the acceleration method further includes: in response to detecting a heating command for the target row or the data table where the target row is located, setting the target row as hot spot data within a second preset time period according to the effective time in the heating command.

[0012] Optionally, in response to detecting a data write operation, the incremental data of the target row is determined based on the pre-write data and post-write data of the target row targeted by the data write operation, including: in response to detecting the data write operation, reading the pre-write data and post-write data of the target row from a data page stored in the disk; comparing the pre-write data and the post-write data to obtain difference data; and compressing the difference data into incremental data.

[0013] Optionally, the acceleration method also includes: in response to detecting a delete operation on a target row, obtaining the life cycle of the target row, wherein the life cycle is the time period from inserting the target row to deleting the target row; when it is determined that the target row is non-hotspot data during the life cycle, writing the most recent version data of the target row before the deletion operation as complete row data into the row buffer pool.

[0014] Optionally, in response to a read request, version data of a target row to be read is calculated in memory based on a row version chain and complete row data, including: in response to the read request, obtaining complete row data from a row buffer pool; obtaining at least one target incremental data from the row version chain, wherein the target incremental data represents incremental data written in an incremental buffer pool after the complete row data is written in the row buffer pool; the at least one target incremental data is arranged in order of write time; and the version data is calculated based on at least one target incremental data and complete row data.

[0015] Another aspect of the present invention provides an acceleration device for hot spot data in a database, comprising: a determination module, for determining, in response to detecting a data write operation, the incremental data of the target row according to the pre-write data and post-write data of the target row targeted by the data write operation; a first write module, for writing the incremental data into a row version chain corresponding to the target row in the incremental buffer pool when it is determined that the registered data of the target row already exists in the row buffer pool, wherein the row buffer pool and the incremental buffer pool are both set in a memory; a second write module, for writing the post-write data into the row buffer pool as the complete row data in response to determining that the target row is hot spot data and when the complete row data of the target row does not exist in the row buffer pool, so as to calculate the version data of the target row to be read according to the row version chain and the complete row data in response to a read request.

[0016] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0017] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0018] Another aspect of the present invention provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when being executed.

[0019] The present invention provides a method for accelerating hot data in a database: in response to detecting a data write operation, the incremental data of the target row is determined according to the pre-write data and post-write data of the target row targeted by the data write operation; when it is determined that the registered data of the target row already exists in the row buffer pool, the incremental data is written into the row version chain corresponding to the target row in the incremental buffer pool, wherein the row buffer pool and the incremental buffer pool are both set in the memory; in response to determining that the target row is hot data, when the complete row data of the target row does not exist in the row buffer pool, the post-write data is written into the row buffer pool as the complete row data. The problem of high memory usage is solved by setting a row-level row buffer pool and an incremental buffer pool, and the technical effect of reducing memory usage is achieved. By using the row-level row buffer pool and the incremental buffer pool in combination, the production of historical version data is completed in the memory, and there is no need to read the version data from the disk, thereby achieving the technical effect of accelerating hot data and reducing access delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0021] Figure 1 An exemplary system architecture is shown to which the method for accelerating hotspot data in a database of the present invention can be applied.

[0022] Figure 2 A flow chart of a method for accelerating hotspot data in a database according to an embodiment of the present invention is shown.

[0023] Figure 3 A schematic diagram of data multiplexing of a target row according to an embodiment of the present invention is shown.

[0024] Figure 4 A schematic diagram showing calculation version data according to an embodiment of the present invention is shown.

[0025] Figure 5 A block diagram of a device for accelerating hotspot data in a database according to an embodiment of the present invention is shown.

[0026] Figure 6 A block diagram of an electronic device suitable for implementing a method for accelerating hotspot data in a database according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0032] In the embodiment of the present invention, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.

[0033] There are three main problems in accelerating database hot data: First, because the data in the buffer pool is stored in pages, a page can contain multiple rows of data, but only a few rows may be hot data. Therefore, loading the entire page into the buffer pool will result in a waste of memory. Second, for multi-version storage, when the historical version of a row is activated as hot data, if it is necessary to roll back the unexecuted (undo) log multiple times to generate the historical version, disk I / O overhead will be generated. Third, there is a lack of multi-dimensional means to activate the hot attributes of the data that users need to access, especially in scenarios where users clearly need to access a certain part of the data frequently. If the hot and cold data are only judged when the user actually accesses it, it is inevitable that there will be obvious access delays in the early stage of application access.

[0034] In order to solve the above technical problems, the present invention proposes a new acceleration method for database hot data. In the above hot data access scenario, it not only ensures the accuracy of data reading, but also reduces the delay and blocking effect when database users access hot data. In order to solve the problem that page memory occupies a lot of memory overhead, the present invention designs a row-level data storage method, and realizes row-level data storage in memory through the designed incremental buffer pool and row buffer. For historical hot data in multi-version storage scenarios, the incremental information of row data changes is memorized through the incremental buffer pool, and the production of historical versions is all completed in memory. At the same time, data that does not have visibility is deleted in time. In addition, for the judgment of the thermal attributes of data, the present invention also provides a multi-dimensional operation method, which can be technically preheated in advance by database maintenance personnel (DBA), or automatically determined by a preset algorithm during the access process.

[0035] Figure 1 An exemplary system architecture is shown to which the method for accelerating hotspot data in a database of the present invention can be applied.

[0036] like Figure 1 As shown, the memory 101 in the system architecture 100 includes a row buffer pool M1 and an incremental buffer pool M2 at the row level in addition to the existing buffer pool M3 for storing in units of pages. The row buffer pool M1 and the incremental buffer pool M2 respectively maintain the management information and data information of the row. The memory of the row buffer pool M1 and the incremental buffer pool M2 is isolated from the memory of the buffer pool M3.

[0037] When the database initializes the buffer pool M3, the row buffer pool M1 and the incremental buffer pool M2 are initialized synchronously. The size of the memory occupied by the row buffer pool M1 and the incremental buffer pool M2 can be configured and isolated from the memory occupied by the buffer pool M3.

[0038] For example, an independent memory area can be opened in the shared memory 101, and multiple memory numbers, such as mid numbers, can be allocated to facilitate the database system to read and write data in the two buffer pools. Similar to the initialization of buffer pool M3, the results of the initialization of row buffer pool M1 and incremental buffer pool M2 directly affect the initialization of the database. If the initialization fails, it is considered that the database instance initialization has failed. The corresponding failure information is recorded by the initialization process in the database operation log for DBA inspection and analysis. Disk 102 can interact with memory 101 through input / output.

[0039] In one embodiment, the database system can read data pages from the disk 102 and store them in the buffer pool M3, so as to filter and screen the data at the page level in the buffer pool M3 to obtain the data corresponding to the target row. In another embodiment, the database system can read the data of the target row from the disk 102, obtain the incremental data of the target row through real-time calculation, and store it in the incremental buffer pool M2; for the target row belonging to hot data, the complete row data of the target row can be stored in the row buffer pool M1.

[0040] It should be noted that the acceleration method for database hotspot data provided in the embodiment of the present invention can generally be executed by a database system. Accordingly, the acceleration device for database hotspot data provided in the embodiment of the present invention can generally be arranged in a database system. The database system can be a single server or a server cluster including multiple servers. The acceleration method for database hotspot data provided in the embodiment of the present invention can also be executed by other servers or server clusters that are different from the above-mentioned servers and can communicate with the terminal device and / or the above-mentioned servers. Correspondingly, the acceleration device for database hotspot data provided in the embodiment of the present invention can also be arranged in other servers or server clusters that are different from the above-mentioned servers and can communicate with the terminal device and / or the above-mentioned servers for execution.

[0041] Figure 2 A flow chart of a method for accelerating hotspot data in a database according to an embodiment of the present invention is shown.

[0042] like Figure 2 As shown, the embodiment 200 includes operations S210 to S230.

[0043] In operation S210 , in response to detecting a data write operation, incremental data of a target row is determined according to pre-write data and post-write data of the target row targeted by the data write operation.

[0044] According to an embodiment of the present invention, a user may perform a modification operation on a target row through a database system, and the database system usually updates the user's modification of the target row to a disk of the database through a data write operation to ensure data consistency.

[0045] Detecting a data write operation means detecting that the user has written a structured query statement (SQL). Therefore, after detecting a data write operation, the pre-write data and post-write data of the target row before and after the data write operation can be obtained in real time, and the differential content before and after the target row is updated can be calculated in real time to obtain the incremental data for the target row.

[0046] In operation S220, when it is determined that the registration data of the target row already exists in the row buffer pool, the incremental data is written into the row version chain corresponding to the target row in the incremental buffer pool, wherein both the row buffer pool and the incremental buffer pool are set in the memory.

[0047] According to an embodiment of the present invention, the registration data is the row information of the target row registered in the row buffer pool. Before writing the incremental data of the target row into the incremental buffer pool, the row information of the data needs to be registered in the row buffer pool to ensure the existence of the data management structure, so that the data in the row buffer pool and the incremental buffer pool can be associated based on the identifier of the target row.

[0048] In one embodiment, the registration data may be the row number of the target row, and whether the target row has been registered is determined by determining whether the row number of the target row already exists in the row buffer pool. When it is determined that the registration data of the target row already exists in the row buffer pool, the incremental data may be directly written into the row version chain corresponding to the target row in the incremental buffer pool. When it is determined that the registration data of the target row does not exist in the row buffer pool, the target row may be registered in the row buffer pool according to the row number of the target row, and after the registration is completed, the incremental data may be written into the row version chain corresponding to the target row in the incremental buffer pool.

[0049] The incremental buffer pool may include multiple data rows and multiple incremental data of each data row. For example, the incremental buffer pool may record multiple incremental data for each data row in the form of a row version chain, and the multiple incremental data are arranged based on the order of the corresponding data write operations.

[0050] The row buffer pool may include multiple data rows and complete row data for each data row.

[0051] Each operation of a user on a data row will result in a version of data. Multiple operations on the data row will result in multiple versions of data. The difference between two adjacent versions of data is the incremental data. The incremental buffer pool is used to record multiple incremental data for each data row, and the row buffer pool is used to record a complete row of data for each data row. Therefore, by using the row buffer pool and the incremental buffer pool together, the complete row data and incremental data can be calculated in memory to obtain multiple versions of data for each data row.

[0052] Different from operations based on executed (redo) or unexecuted (undo) logs, the incremental data of the embodiment of the present invention stores the difference data of each row before and after the user operation. In addition, when calculating the incremental data of the target row, the present invention only reads the pre-write data and post-write data of the target row from the disk, without reading the data page where the target row is located in the disk into the memory and obtaining the pre-write data and post-write data of the target row through filtering operations. Therefore, compared with the original page-level or log operations, only recording the incremental data of the version on the row can meet the real-time storage requirements, while also reducing memory overhead.

[0053] In operation S230, in response to determining that the target row is hot data, if the complete row data of the target row does not exist in the row buffer pool, the written data is written into the row buffer pool as complete row data, so as to respond to the read request and calculate the version data of the target row to be read in memory according to the row version chain and the complete row data.

[0054] Different from the data storage engine actively writing pages from disk to page-level buffer pool as needed during SQL execution, the storage of complete row data in the row buffer pool in the embodiment of the present invention is for row-level hot spot data.

[0055] When the target row is determined to be hot data, the complete row data is actively loaded and the hot and cold characteristics of the target row are recorded in the row buffer pool. In addition, since multiple versions of the target row data can be calculated through one complete row data + at least one incremental data, multiple versions of the data can be calculated by ensuring that one complete row data of the target row exists in the row buffer pool.

[0056] Therefore, for the target row determined to be hot data, before performing the write operation in the row buffer pool, it is necessary to determine whether a complete row of data of the target row already exists in the row buffer pool. If so, there is no need to read the complete row data from the disk through a new I / O operation and write it into the memory; if not, the written data is written into the row buffer pool as the complete row data.

[0057] After completing the write operations of the above row buffer pool and incremental buffer pool, if subsequent users need to read the latest version data / historical version data of the target row, they can calculate it based on the row version chain and the complete row data without having to read it from the disk, thereby accelerating hot data and reducing the impact of access delays on users.

[0058] For example, take the target row as row1 and hot data as an example. When the user performs the first data write operation on row1, the incremental data of this time can be written in the row version chain of row1 in the incremental buffer pool. Since there is no complete row data in the row buffer pool after the first data write operation, the data after the write operation of this data write operation can be written into the row buffer pool as complete row data. In the subsequent two data write operations, only two new incremental data are written in the row version chain of row1 in the incremental buffer pool. When the user initiates a read request, the version data can be obtained based on the complete row data + the incremental data stored twice.

[0059] The embodiment of the present invention provides a method for accelerating hot data in a database: in response to detecting a data write operation, the incremental data of the target row is determined according to the pre-write data and post-write data of the target row targeted by the data write operation; when it is determined that the registered data of the target row already exists in the row buffer pool, the incremental data is written into the row version chain corresponding to the target row in the incremental buffer pool, wherein the row buffer pool and the incremental buffer pool are both set in the memory; in response to determining that the target row is hot data, when the complete row data of the target row does not exist in the row buffer pool, the post-write data is written into the row buffer pool as the complete row data. In this embodiment, the problem of high memory usage is solved by setting a row-level row buffer pool and an incremental buffer pool, and the technical effect of reducing memory usage is achieved. By using the row-level row buffer pool and the incremental buffer pool together, the production of historical version data is completed in the memory, and there is no need to read the version data from the disk, thereby achieving the technical effect of accelerating hot data and reducing access delay.

[0060] In one embodiment, the data structure of the incremental buffer pool may be the belonging row information + incremental data, that is, the incremental data and the belonging row information are synchronously written into the row version chain corresponding to the target row.

[0061] In another embodiment, the acceleration method further includes: when writing the incremental data into the row version chain corresponding to the target row, synchronously writing the target row's belonging row information, index information and data visible range into the row version chain.

[0062] The index information is used to query the incremental data. The data visible range is an attribute of the incremental data, which represents the range of the incremental data of the target row that can be read.

[0063] For example, different data structures can be constructed to synchronously carry the row, index information, and data visibility range of the incremental data when writing the incremental data to the incremental buffer pool. For example, this can be achieved by constructing a red-black tree, a bidirectional linked list, a skip list, etc.

[0064] The embodiments of the present invention can maintain the mapping association of rows, stored incremental data, and indexes by synchronously writing the belonging row information, index information, and data visible range into the row version chain, so as to realize fast data retrieval.

[0065] In one embodiment, the data visibility range can be used as a standard for releasing memory in the incremental buffer pool. Specifically, the acceleration method also includes: determining the validity of the incremental data for each data write operation according to the data visibility range; deleting invalid incremental data in the row version chain of the incremental buffer pool; and deleting the complete row data from the row buffer pool when the incremental data of each data write operation in the row version chain are invalid.

[0066] According to an embodiment of the present invention, the row version chain includes incremental data of at least one data writing operation and is sorted according to the writing time of the data writing operation.

[0067] In one embodiment, in a multi-version storage and concurrent reading scenario, row-level read-write lock protection can be used on the row version chain of the incremental buffer pool, which supports concurrent reading and avoids read-write data conflicts, thereby ensuring data accuracy in concurrent reading scenarios.

[0068] The method for determining the validity of incremental data includes: determining that the incremental data is invalid when the data visibility range indicates that it is globally invisible; and determining that the incremental data is valid when the data visibility range indicates that it is globally visible.

[0069] In a multi-version storage scenario, if the incremental data used to calculate the current version data is globally invisible, it means that the incremental data is invalid and the user does not need to query the current version data. Therefore, the incremental data can be deleted from the row version chain, that is, the incremental data can be removed from the incremental buffer pool to reduce invalid memory usage.

[0070] The complete row data in the row buffer pool is usually used in combination with the incremental data in the incremental buffer pool. When all the incremental data of the target row in the incremental buffer pool is invalid, it is difficult to calculate the version data based on the combination of the above two buffer pools. Therefore, when all the incremental data is invalid, it is necessary not only to delete all the invalid incremental data in the incremental buffer pool, but also to delete the complete row data of the target row from the row buffer pool to further reduce invalid memory usage.

[0071] Specifically, the deletion operation of the row buffer pool and the incremental buffer pool can be implemented through the memory recycling mechanism.

[0072] It should be noted that, usually, the hot and cold characteristics of data (hot data / non-hot data) need to be defined after the data enters the storage. Therefore, in order to ensure the continuity and completeness of the hot data, it is necessary to record the data difference between each data write operation of the user and the previous operation to form incremental data storage. For the target row determined as hot data, the complete row data is written to the row buffer pool.

[0073] According to an embodiment of the present invention, determining that the target row is hot data is achieved in the following manner: obtaining the reading frequency of the target row in a first preset time period; and determining that the target row is hot data when the reading frequency is greater than a frequency threshold.

[0074] According to the embodiments of the present invention, considering that the existing method requires the user to actually access the data to determine whether it is hot data, and the first access or the first few accesses usually result in access delays, the embodiments of the present invention advance the determination technology of hot data to the storage stage, and do not require the user to determine whether it is hot data through a read request access.

[0075] Specifically, a frequency statistics function is added to the read interface of the storage layer to obtain the read frequency of the target row in a first preset time period. The first preset time period may be a period before the current data write operation.

[0076] When the read frequency is greater than the frequency threshold, the target row is determined to be hot data; otherwise, it is determined to be non-hot data. For example, it can be implemented according to the Least Frequently Used (LFU) rule: when the read frequency of a row exceeds the set value, the row will be marked as hot data, and the mark is recorded as the attribute value of the hot attribute in the corresponding row data structure in the row buffer pool.

[0077] In another embodiment, the hotspot data determination may be implemented by comparing the number of reads with a number threshold. The comparison method is described in the above-mentioned reading frequency and will not be described in detail here.

[0078] In an embodiment of the present invention, by implementing the determination of hotspot data through a read interface during a data write operation, for a target row belonging to hotspot data, the storage layer can actively load the complete row data when writing the calculated incremental data, and save it in the incremental buffer pool, and the subsequent read operation can complete the read access to the target row in the buffer pool, thereby greatly improving the read performance. Compared with the existing method of performing hotspot determination at access time and loading it in a page-level buffer pool, the embodiment of the present invention can load the complete row data at the row level in advance in the row-level row buffer pool by preheating at read time, thereby accelerating the hotspot data.

[0079] According to an embodiment of the present invention, the acceleration method also includes: when it is determined that the target row is hot data, establishing a daemon thread for the target row to call the daemon thread to calculate the heat value of the target row, and when the heat value continues to be lower than the heat threshold, modifying the hot attribute for the target row in the row buffer pool to non-hot data.

[0080] The daemon thread is used to calculate the heat value of the target row marked as hot data, and based on this, modify / not modify the hot attribute of the target row. That is, for the established read-time warm-up mechanism, the present invention also establishes an automatic cold-hot degradation mechanism accordingly.

[0081] Specifically, the reading frequency in each time interval can be calculated regularly based on the time and the number of reads, and used as the heat value. For example, the time interval can be 1 minute. When the reading frequency is 2 times / min for 3 consecutive minutes, which is lower than the heat threshold 3, the hot attribute of the target row in the row buffer pool is modified to non-hot data. The time interval frequency can be customized (3 consecutive times), and non-hot data can also be called cold data.

[0082] Alternatively, you can also customize the activity factor and multiply the activity factor by the reading frequency to get the final reading frequency to determine whether to modify the hotspot attribute. For example, the activity factor can be 1.5. Taking the above 2 times / min as an example, after adding the activity factor, the reading frequency of 3 times / min for 3 consecutive minutes is equal to the heat threshold of 3, and the hotspot attribute is not modified.

[0083] In this embodiment, by establishing a daemon thread to perform an autonomous hot and cold degradation mechanism and modifying the hot spot attributes in the row buffer pool, when incremental data is subsequently added, the complete row data of the row will no longer be actively loaded, and there is no need to judge whether there is complete row data, thereby further improving the reading performance.

[0084] According to an embodiment of the present invention, the activation of hotspot data can also be achieved by actively preheating the command line. Specifically, the acceleration method also includes: in response to detecting a heating command for a target row or a data table where the target row is located, setting the target row as hotspot data within a second preset time period according to the effective time in the heating command.

[0085] Specifically, the database administrator may generate a heating command through a command line to heat the target row or the data table where the target row is located, and the disk may store the data in the data table through at least one data page.

[0086] The command line includes at least the following parameters: operation type, table name of the data table. Operation types include heating / heating off. It can be understood that the command line with the operation type of heating generates a heating command, and the command line with the operation type of heating off generates a heat degradation command, that is, to change the hotspot attribute in the row buffer pool from hotspot data to non-hotspot data, such as from hot to cold. For example, heating / heating off can be represented by on / off respectively. The table name of the data table is the name when the table is created.

[0087] The operation type and table name in the command line are mandatory and are used to heat / turn off the database table. The command line can also include a row number to heat / turn off the specified target row. For example, if the row number value is empty, the default command scope is the table level, that is, all rows in the partition of the table name are heated.

[0088] In some embodiments, the command line may also include a valid time, which is used to specify the time when the target row / data table is the hot data. The value of the valid time is a number, which is used to represent the effective duration. For example, if the valid time is 10, the second preset time period can be 10 minutes starting from the current time. During this time period, the target row is the hot data. When the value of the valid time is empty, it is permanently valid by default, which can be achieved by setting a default long valid time value, such as 9999999.

[0089] In some embodiments, the command line may also include a filter condition, which is combined with the table name to achieve heating of rows within the data table. The value of the filter condition is in a valid range, for example, 1 to 10 rows. When the value does not meet the condition / is empty, the data table is heated; when the value meets the condition, the heating of at least one row of data corresponding to the value is achieved. For example, taking a certain table XXX as an example, the value of the filter condition is 1 to 10, but the table XXX only includes 1 to 9 rows. Therefore, the value does not meet the condition, and table-level heating is achieved for the table XXX. It can be understood that row number, filter condition, and effective time are all optional options.

[0090] The specific execution of the command is completed by the background thread. For example, according to the parameters passed in the command line, the background thread first converts the table name into the partition information recorded in the storage layer by querying the system library, and then combines the page and slot information recorded in the bitmap and the configured filter conditions to calculate all the row information of the command operation, and at the same time determines whether it is necessary to actively load the complete row data of the row according to the hot spot attributes in the incremental buffer pool.

[0091] In addition, unlike the read-time warm-up mechanism, the data heated by the active warm-up mechanism cannot be downgraded autonomously, but it can be downgraded by providing two solutions: one is to set the effective time so that its hotspot attribute is only valid for a period of time, and after the timeout, it will be converted from hotspot data to non-hotspot data according to the hotspot attribute. The other is active cooling, and the database administrator still turns off heating through the command line.

[0092] In one embodiment, read-time preheating and active preheating are effective separately. If any one of the two meets the conditions, the target row can be set as hot data, and the complete row data of the target row will be loaded into the row buffer pool. The table level has a higher priority than the row level. If table-level preheating is configured, all rows in the table need to be loaded with complete row data by default.

[0093] Unlike the forced loading of complete row data in the slot reuse scenario, the embodiments of the present invention do not throw an exception when encountering an Out Of Memory Error (OOM) problem when preheating or actively preheating to load complete row data during reading. Since the row buffer pool is for hot data, and the disk records the version data of each write operation, the row buffer pool encountering an OOM problem does not affect the data read consistency. Therefore, even if the row buffer pool encounters an OOM problem and does not throw an exception, it can ensure the normal execution of the transaction and avoid affecting the acceleration mechanism and affecting the normal user access.

[0094] According to an embodiment of the present invention, in response to detecting a data write operation, the incremental data of the target row is determined based on the pre-write data and post-write data of the target row targeted by the data write operation, including: in response to detecting the data write operation, reading the pre-write data and post-write data of the target row from a data page stored in a disk; comparing the pre-write data and the post-write data to obtain difference data; and compressing the difference data into incremental data.

[0095] In this embodiment, after receiving the write information of the target row to be updated transmitted by the SQL layer, the storage layer can read the pre-write data and post-write data of the target row from the "page" stored on the disk, and compare the pre-write data and post-write data to obtain differential data. In one embodiment, the differential data obtained by comparison can be directly used as incremental data.

[0096] In another embodiment, the difference data may be further compressed to obtain incremental data, so as to reduce the memory usage of the incremental buffer pool.

[0097] According to an embodiment of the present invention, the acceleration method also includes: in response to detecting a delete operation on a target row, obtaining the life cycle of the target row, wherein the life cycle is the time period from inserting the target row to deleting the target row; when it is determined that the target row is non-hotspot data during the life cycle, writing the most recent version data of the target row before the deletion operation as complete row data into the row buffer pool.

[0098] A life cycle of a target row refers to the period from an insert operation to a delete operation on the data in the row. The incremental data in the incremental buffer pool is also the incremental data that occurs within a life cycle.

[0099] Although the row buffer pool is for hot data, that is, as mentioned above, for hot data, when the row buffer pool already has complete row data, it will not continue to write, and when there is no complete row data, it will be written once and will not be written again. However, for non-hot data, the row buffer pool must also be forced to load a complete row of data once in a life cycle.

[0100] For example, for non-hot data, the complete row data of the target row does not need to be stored in real time, but it needs to be forcibly stored at the end of a life cycle, that is, when the page slot in the disk is reused, to avoid the loss of historical row data.

[0101] For example, when a delete operation is detected for a target row, the current life cycle of the target row needs to be ended. If the target row is non-hot data throughout its life cycle, the latest version of the data before the delete operation is forcibly written into the row buffer pool as complete row data before the life cycle ends, to ensure that there is a complete row of data within one life cycle. The latest version of the data is the data after the last data write operation.

[0102] Figure 3 A schematic diagram of data multiplexing of a target row according to an embodiment of the present invention is shown.

[0103] like Figure 3 As shown, the row buffer pool M1 may include multiple rows, such as row 1, row 2, row 3, row 4, row 5 to row N. For the target row 3011, such as row 4, the row buffer pool M1 already has its registered data. If row 4 is also hot data, the row buffer pool M1 also includes a complete row data of row 4.

[0104] The incremental buffer pool M2 includes the row version chain of row 4 to row N. For example, the row version chain of row 4 is 3021. In the life cycle of row 4, there is an insert operation, such as INS, three data write operations 1 to 3, namely UPD1 to UPD3, and a delete operation, such as DEL. The incremental data between the five operations can be recorded through the row version chain 3021.

[0105] According to an embodiment of the present invention, in response to a read request, version data of a target row to be read is calculated based on a row version chain and complete row data, including: in response to the read request, obtaining complete row data from a row buffer pool; obtaining at least one target incremental data from the row version chain, wherein the target incremental data represents incremental data written in an incremental buffer pool after the complete row data is written in the row buffer pool; the at least one target incremental data is arranged in order of write time; and the version data is calculated based on at least one target incremental data and complete row data.

[0106] Figure 4 FIG. 2 shows a schematic diagram of calculating version data according to an embodiment of the present invention. Figure 4 As shown in the figure, the complete row data, such as full, is read from the row buffer pool and is globally visible. After a delete operation, such as DEL, occurs, the complete row data is not globally visible. In a life cycle, an insert operation, such as INS, three data write operations 1 to 3, such as UPD1 to UPD3, and a delete operation, such as DEL, occur. In addition, v0 to v4 are incremental data.

[0107] The following is the version data obtained for each operation based on the complete row data and incremental data: the delete operation corresponds to global invisibility and no version data, the write operation 1 corresponds to full+v0, the write operation 2 corresponds to full+v0+v1, the write operation 3 corresponds to full+v0+v1+v2, and the insert operation corresponds to full+v0+v1+v2+v3.

[0108] In addition, it should be noted that the same row of incremental data only requires one complete row of data. The row data is not bound to hot or cold attributes, and its life cycle is consistent with the incremental data of the corresponding row.

[0109] In summary, the embodiments of the present invention provide a method for accelerating hot spot data in a database, the goal of which is to reduce the impact of delays and blocking when database users access hot spot data. Specifically, for scenarios where page memory occupies a large amount of memory overhead, the problem of large memory usage is solved through row-level row buffer pools and incremental buffer pools. For the problem of multi-version storage, the production of historical version data is completed entirely in memory through the combination of row-level row buffer pools and incremental buffer pools, and data that does not have global visibility is deleted in a timely manner. Multi-dimensional operating methods are provided for determining the hot spot attributes of data: active preheating by database maintenance personnel (DBA) and read-time preheating automatically determined by a preset algorithm.

[0110] Figure 5 A block diagram of a device for accelerating hotspot data in a database according to an embodiment of the present invention is shown.

[0111] like Figure 5As shown, the acceleration device 500 for database hotspot data includes a determination module 510 , a first writing module 520 , and a second writing module 530 .

[0112] The determination module 510 is used to determine the incremental data of the target row according to the pre-write data and post-write data of the target row targeted by the data write operation in response to detecting the data write operation. The first write module 520 is used to write the incremental data into the row version chain corresponding to the target row in the incremental buffer pool when it is determined that the registered data of the target row already exists in the row buffer pool, wherein the row buffer pool and the incremental buffer pool are both set in the memory. The second write module 530 is used to write the post-write data into the row buffer pool as the complete row data in response to determining that the target row is hot data and when the complete row data of the target row does not exist in the row buffer pool, so as to calculate the version data of the target row to be read according to the row version chain and the complete row data in response to the read request.

[0113] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.

[0114] For example, any multiple of the determination module 510, the first writing module 520, and the second writing module 530 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the determination module 510, the first writing module 520, and the second writing module 530 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the determination module 510, the first writing module 520, and the second writing module 530 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0115] It should be noted that the device part in the embodiment of the present invention corresponds to the method part in the embodiment of the present invention. The description of the device part specifically refers to the method part, which will not be repeated here.

[0116] Figure 6 A block diagram of an electronic device suitable for implementing a method for accelerating hotspot data in a database according to an embodiment of the present invention is shown. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0117] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0118] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0119] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0120] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0121] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0122] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0123] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or the RAM 603 described above and / or one or more memories other than the ROM 602 and the RAM 603 .

[0124] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0125] When the computer program is executed by the processor 601, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0126] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0127] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0129] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for accelerating hotspot data in a database, characterized in that: The acceleration method comprises: In response to detecting a data write operation, determining incremental data of the target row according to pre-write data and post-write data of the target row targeted by the data write operation; When it is determined that the registration data of the target row already exists in the row buffer pool, writing the incremental data into the row version chain corresponding to the target row in the incremental buffer pool, wherein both the row buffer pool and the incremental buffer pool are set in the memory; In response to determining that the target row is hot data, if the complete row data of the target row does not exist in the row buffer pool, the written data is written into the row buffer pool as the complete row data, so as to respond to the read request and calculate the version data of the target row to be read according to the row version chain and the complete row data.

2. The acceleration method according to claim 1, characterized in that: The acceleration method further comprises: When the incremental data is written into the row version chain corresponding to the target row, the row information, index information and data visible range of the target row are synchronously written into the row version chain.

3. The acceleration method according to claim 2, characterized in that: The row version chain includes the incremental data of at least one data writing operation; The acceleration method further comprises: For the incremental data of each data writing operation, determining the validity of the incremental data according to the data visible range; Deleting the invalid incremental data in the row version chain of the incremental buffer pool; In the case where the incremental data of each data write operation in the row version chain is invalid, the complete row data is deleted from the row buffer pool.

4. The acceleration method according to claim 1, characterized in that: The determination that the target row is hot data is achieved in the following manner: Obtaining a reading frequency of the target row in a first preset time period; When the reading frequency is greater than a frequency threshold, it is determined that the target row is hotspot data.

5. The acceleration method according to claim 4, characterized in that: The acceleration method further comprises: When it is determined that the target row is hot data, a daemon thread is established for the target row to call the daemon thread to calculate the heat value of the target row, and when the heat value continues to be lower than the heat threshold, the hot attribute of the target row in the row buffer pool is modified to non-hot data.

6. The acceleration method according to claim 4, characterized in that: The acceleration method further comprises: In response to detecting a heating command for the target row or the data table where the target row is located, the target row is set as hot spot data within a second preset time period according to the effective time in the heating command.

7. The acceleration method according to claim 1, characterized in that: In response to detecting the data write operation, determining the incremental data of the target row according to the pre-write data and the post-write data of the target row targeted by the data write operation, comprises: In response to detecting a data write operation, reading the pre-write data and the post-write data of the target row from a data page stored in a disk; Comparing the pre-writing data with the post-writing data to obtain difference data; The difference data is compressed into the incremental data.

8. The acceleration method according to claim 1, characterized in that: The acceleration method further comprises: In response to detecting a delete operation on the target row, acquiring a life cycle of the target row, wherein the life cycle is a time period from inserting the target row to deleting the target row; When it is determined that the target row is non-hotspot data during the life cycle, the latest version data of the target row before the deletion operation is written into the row buffer pool as complete row data.

9. The acceleration method according to claim 1, characterized in that: The step of calculating, in response to the read request, version data of the target row to be read according to the row version chain and the complete row data includes: In response to a read request, acquiring the complete row data from the row buffer pool; Acquire at least one target incremental data from the row version chain, wherein the target incremental data represents incremental data written in the incremental buffer pool after the complete row data is written in the row buffer pool; at least one target incremental data is arranged in order of writing time; The version data is calculated based on at least one of the target incremental data and the complete row data.

10. An electronic device comprising: one or more processors; a memory for storing one or more programs, It is characterized in that when the one or more programs are executed by the one or more processors, the one or more processors implement the acceleration method described in any one of claims 1 to 9.

Images