Method and device for data management optimization

Abstract

The present invention discloses a method and device for data management optimization, which includes: initializing the acceleration card storage, host cache page information, the host page cache and the acceleration card page management information according to the page attribute data structure, the capacity of the acceleration card storage and the capacity of the host page cache; receiving and sending an operation instruction to the accelerator, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, forming dirty page data, and returning the processing result to the host; and simultaneously returning dirty page modification information corresponding to the operation instruction, caching the dirty page data in the host page cache, and directly reading the dirty page data from the host page cache when the dirty page data is written back, thereby reducing the communication overhead of data interaction and improving the data processing performance and reliability of the database system.

Description

Technical Field

[0001] The present application relates to the field of computer and database technology, and more specifically, to a method and device for optimizing data management. Background Art

[0002] Database management systems in the big data era face the need to process and analyze massive amounts of data, and hardware acceleration is often used to meet performance requirements. These acceleration solutions typically add a PCIe accelerator card to the server where the database is deployed. This improves database processing performance by offloading some database operations to the accelerator card.

[0003] However, the resulting PCIe interface processing latency significantly impacts overall acceleration performance. To reduce latency, one approach is to further reduce the processing latency of the PCIe interface itself. The industry has proposed a series of PCIe-based cache coherence protocol interfaces, such as CXL and CCIX, to reduce interface latency, but this does not eliminate it. A second approach is to minimize the number of data interactions on the interface. Typically, accelerator cards are equipped with large amounts of RAM (main memory, the memory storage that directly exchanges data with the CPU) to ensure that data access and processing are completed on the accelerator card as much as possible, with only the final results returned to the server host. However, due to the explosive growth in data volume, regardless of the amount of RAM installed on the accelerator card, especially in OLTP-type database management systems, the acceleration function on the accelerator card faces the problem of page misses on the accelerator card causing swapping in and out, and the writing of modified and updated data pages back to the server host. This increases the number of data interactions and impacts the acceleration effect.

[0004] Existing accelerator cards and CPUs communicate via PCIe, which carries significant communication overhead. Using DMA (Direct Memory Access), the CPU only initiates transfers and does not participate in data copying. The data copying or data transfer from one address space to another is performed and completed by the hardware DMA controller. While DMA can improve data throughput, data transmission latency remains high, hindering the performance of accelerator card applications. Consequently, several industry alliances are developing new transaction layers and interface protocols based on the PCIe interface to reduce communication latency between accelerator cards and CPUs. CXL is one such implementation, achieving this goal through a built-in cache coherence protocol and unified pooling of CPU and accelerator card memory. While cache coherence protocols significantly reduce latency on the PCIe interface, interface latency still persists. This additional latency introduced by the accelerator card offsets some of the gains brought by the accelerator card, still impacting overall system performance. Therefore, mitigating the performance impact of data writeback is a pressing issue.

[0005] Based on this, it is necessary to introduce a new method and device that can reduce the number of interface-based data interactions between the accelerator card and the host on the basis of ensuring data cache consistency, and solve technical problems such as frequent data interaction, data transmission delay and PCIe interface delay in the existing technology, thereby realizing acceleration of the database system processing performance based on the accelerator card, reducing the communication overhead caused by data interaction, improving the performance of data transmission, data write back and data processing, and the applicability of the accelerator card. Summary of the Invention

[0006] In response to the technical problems mentioned above, the present invention provides a method and device for data management optimization. When an accelerator card returns the processing result of an operation instruction, it also returns page data corresponding to the processing result of the operation instruction, and completes caching on the host in advance for the corresponding user's query request, data persistent storage and data recovery. On the basis of ensuring data cache consistency, the number of interface-based data interactions between the accelerator card and the host is greatly reduced, and technical problems such as frequent data interaction, data transmission delay and PCIe interface delay in the prior art are solved, thereby realizing acceleration of the database system processing performance based on the accelerator card, reducing the communication overhead caused by data interaction, and improving the data processing performance and reliability of the database system, as well as the applicability of the accelerator card.

[0007] The present invention provides a method for optimizing data management, comprising:

[0008] S101, defining a page attribute data structure, and initializing the accelerator card storage, host cache page information, the host page cache, and accelerator card page management information based on the page attribute data structure, the capacity of the accelerator card storage, and the capacity of the host page cache; S102, receiving and sending an operation instruction to the accelerator, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, generating dirty page data, and returning the processing result to the host; S103, the host writing the dirty page data back to the host storage based on the dirty page mark and the processing result;

[0009] In which, when the accelerator returns the processing result to the host, it also returns the dirty page modification information corresponding to the operation instruction. The host caches the dirty page data in the host page cache according to the dirty page modification information. When the dirty page data is written back to the host storage, the dirty page data is directly read from the host page cache; the page attribute data structure includes the database name, data table number, host side page number, accelerator card side page number, dirty page mark, page position and tuple information; the types of the operation instructions include add operation instructions, delete operation instructions, modify operation instructions, and query operation instructions. The operation instructions are obtained by parsing and optimizing the user's database operation request or generated by the auxiliary process of the host, wherein the operation instructions generated by the auxiliary process of the host do not include the query operation instructions.

[0010] As described above, the S101, defining the page attribute data structure, and initializing the acceleration card storage, host cache page information, the host page cache and the acceleration card page management information according to the page attribute data structure, the capacity of the acceleration card storage and the capacity of the host page cache, includes: initializing the acceleration card storage and the host cache page information, loading the data page from the persistent storage of the host and storing it in the acceleration card storage according to the page attribute data structure and the capacity of the acceleration card storage, generating the acceleration card side page number, and updating the host cache page information, including the database name, the data table number, the data page number, the dirty page mark, the page position and the tuple information of the loaded data page; initializing the host page cache and the acceleration card page management information, and setting the host page cache and the acceleration card page management information to empty according to the initialized acceleration card storage and the host cache page information; wherein the initialized dirty page marks are all empty values, and when the data page is modified to form a dirty page, the dirty page mark is set to a dirty page.

[0011] As described above, the step S102 of receiving and sending an operation instruction to the accelerator, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, forming dirty page data, and returning the processing result to the host includes the step of processing the add, delete, and modify operation request, specifically: the host receives and sends the operation instruction to the accelerator; when the type of the operation instruction received by the accelerator from the host is the add operation instruction, or the delete operation instruction, or the modify operation instruction, or the operation instruction generated by the auxiliary process of the host, the host will parse and optimize the first execution instruction obtained by the operation instruction. The plan is sent to the accelerator, and the accelerator processes the data page according to the first execution plan and the page attribute data structure, sets the dirty page mark of the processed data page to a dirty page to form the dirty page data, updates the host cache page information, and records the dirty page modification information, including the accelerator card side page number of the dirty page, the database name, the data table number, the page position and the tuple information; the accelerator returns the processing result to the host and returns the dirty page modification information to the host; the host caches the dirty page data and the dirty page modification information according to the dirty page modification information and the host cache page information.

[0012] As described above, the steps of the host caching the dirty page data and the dirty page modification information according to the dirty page modification information and the host cache page information are as follows: if the host cache page information does not have the accelerator card side page number of the dirty page, the dirty page modification information is stored in the host cache page information and the host side page number is assigned, and the data page and metadata corresponding to the dirty page are read from the accelerator according to the dirty page modification information, and cached in the host page cache; if the host cache page information has the accelerator card side page number of the dirty page, the dirty page modification information is updated to the host cache page information, and according to the host side page number of the dirty page, the accelerator card side page number, the page position and the tuple information, the metadata corresponding to the dirty page is read from the accelerator and updated to the host page cache, and data offset processing is performed on the data page corresponding to the dirty page in the host page cache.

[0013] As described above, the step of receiving and sending an operation instruction to the accelerator, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, forming dirty page data, and returning the processing result to the host includes the step of query operation request processing, specifically: the host receives the query operation instruction of the user, parses and optimizes the query operation instruction to obtain a second execution plan corresponding to the query operation instruction; according to the second execution plan and the host cache page information, if the query result corresponding to the second execution plan exists in the host page cache, and the second execution plan is in the host's If the execution cost is less than or equal to the execution cost of the second execution plan on the accelerator, the second execution plan is executed on the host, and the query result is obtained and returned from the host page cache according to the second execution plan; if the query result corresponding to the second execution plan does not exist in the host page cache, or the query result corresponding to the second execution plan exists in the host page cache but the execution cost of the second execution plan on the host is greater than the execution cost of the second execution plan on the accelerator, the second execution plan is sent to the accelerator, and the accelerator obtains and returns the query result from the acceleration card storage of the accelerator according to the second execution plan.

[0014] As described above, S102 receives and sends an operation instruction to the accelerator, and the step of the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure also includes the step of processing the data page fault, specifically: when the data page does not exist in the acceleration card storage, the accelerator sends a page fault request to the host; the host parses the operation instruction to obtain the database name and the data table number according to the page fault request, and loads the data page corresponding to the operation instruction from the host's persistent storage according to the database name and the data table number, and allocates the acceleration card side page number; according to the capacity of the acceleration card storage and the capacity of the host page cache, the data page corresponding to the operation instruction is replaced.

[0015] Optionally, the step of performing page replacement processing on the data page corresponding to the operation instruction according to the storage capacity of the accelerator card and the capacity of the host page cache includes host-side page replacement processing and accelerator-side page replacement processing;

[0016] in,

[0017] The steps of the host-side page replacement processing are as follows: 1) when the available capacity of the host page cache is greater than or equal to the data volume of the data page corresponding to the operation instruction, caching the data page corresponding to the operation instruction in the host page cache, assigning the host-side page number to the data page corresponding to the operation instruction, storing the host-side page number, the accelerator card-side page number, the database name, and the data table number in the host cache page information, and initializing the dirty page mark to a null value; sending the data page corresponding to the operation instruction to the accelerator; when the available capacity of the host page cache is less than the data volume of the data page corresponding to the operation instruction, the host sending the data page corresponding to the operation instruction to the accelerator;

[0018] The accelerator-side page replacement processing steps are as follows: the accelerator receives the data page corresponding to the operation instruction and the accelerator card-side page number, and stores the data page corresponding to the operation instruction according to the available capacity of the accelerator card storage; 1) when the available capacity of the accelerator card storage is greater than or equal to the data volume of the data page corresponding to the operation instruction, the data page corresponding to the operation instruction is stored in the accelerator card storage; the accelerator card-side page number, the database name, and the data table number are respectively updated to the host cache page information, and the dirty page mark is initialized to a null value. 2) when the available capacity of the accelerator card storage is less than the data volume of the data page corresponding to the operation instruction, the stored data page is deleted from the accelerator card storage according to a preset data page replacement algorithm and the data volume of the data page corresponding to the operation instruction, and the data page corresponding to the operation instruction is stored in the accelerator card storage; the accelerator card-side page number, the database name, and the data table number are respectively updated to the host cache page information, and the dirty page mark is initialized to a null value.

[0019] The preset data page replacement algorithm may be set according to the residence time, access times, weight and data size of the data page.

[0020] As described above, the step of writing the dirty page data back to the host storage according to the dirty page mark and the processing result in S103 includes: storing the dirty page data in the host page cache to the persistent storage of the slave host according to the dirty page write-back policy and the dirty page mark of the data page, and setting the dirty page mark of the data page to a null value;

[0021] The dirty page write-back policy is set according to the residence time, polling order, data size and modification times of the dirty page data in the host page cache.

[0022] Optionally, after writing the dirty page data back to the host storage according to the dirty page mark, the step of accelerator fault recovery is also included, specifically: when the accelerator fails, the host receives and processes the data page according to the type of the operation instruction and the page attribute data structure, adds a dirty page mark to the processed data page, and returns the processing result; the host restores the dirty page data to the acceleration card storage of the accelerator according to the log information of the accelerator, and loads the data page from the persistent storage of the host to the acceleration card storage according to the capacity of the acceleration card storage, generates the acceleration card side page number, and updates the host cache page information on the accelerator; after the accelerator returns to normal, it resumes receiving the operation instructions issued by the accelerator to receive the host.

[0023] Correspondingly, the present invention also provides a data management optimization device, which is applied to a database system connected to an accelerator. The device includes a parsing optimization module, an interface processing module, an accelerator card page management module, a page cache module, an accelerated execution module, an accelerator card storage module, and a host cache page information management module.

[0024] The parsing and optimization module, the interface processing module, the accelerator card page management module, and the page cache module are deployed in the host, and the acceleration execution module, the accelerator card storage module, and the host cache page information management module are deployed in the accelerator.

[0025] The parsing and optimization module is used to receive a user's database operation request, and parse and optimize the user's database operation request to obtain an execution plan corresponding to the user's database operation request;

[0026] The interface processing module is configured to send an execution plan corresponding to the user's database operation request to the accelerator;

[0027] The accelerator card page management module is configured to cache the accelerator card side page number, the database name, the data table number, the data page number, the dirty page flag, the in-page position, and the tuple information corresponding to the dirty page data from the host cache page information management module when the accelerator processes the data page according to the type of the operation instruction and the page attribute data structure and returns the processing result, and generate and store the host side page number; and when performing the host side page replacement processing on the data page corresponding to the operation instruction according to the capacity of the host page cache, assign the host side page number to the data page corresponding to the operation instruction, and store the host side page number, the accelerator card side page number, the database name, the data table number, and the dirty page flag;

[0028] The page cache module is configured to cache the dirty page data in the acceleration card storage module, and cache the data page corresponding to the operation instruction when performing the host-side page replacement processing on the data page corresponding to the operation instruction according to the capacity of the host page cache;

[0029] The acceleration execution module is configured to receive and execute the execution plan corresponding to the user's database operation request issued by the interface processing module and send it to the accelerator;

[0030] The accelerator card storage module is configured to load and store the data page from the host's persistent storage according to the page attribute data structure and the accelerator card storage capacity, and generate the accelerator card-side page number;

[0031] The host cache page information management module is used to store and update the host cache page information on the accelerator, including the database name, the data table number, the data page number, the dirty page mark, the page position and the tuple information of the loaded data page, and generate and store the accelerator card side page number.

[0032] By applying the above technical solution, when the accelerator returns the processing result to the host, it also returns the dirty page modification information corresponding to the operation instruction. The host caches the dirty page data modified according to the dirty page modification information in the host page cache. When the dirty page data is written back to the host storage, the dirty page data is directly read from the host page cache. This enables the accelerator card to return the page data corresponding to the processing result of the operation instruction at the same time as returning the processing result of the operation instruction, and completes the cache on the host in advance for the corresponding user's query request, data persistent storage and data recovery. On the basis of ensuring data cache consistency, the number of interface-based data interactions between the accelerator card and the host is greatly reduced, and technical problems such as frequent data interaction, data transmission delay and PCIe interface delay in the prior art are solved, thereby completing the acceleration of the database system processing performance based on the accelerator card, reducing the communication overhead caused by data interaction, and improving the data processing performance and reliability of the database system, as well as the applicability of the accelerator card. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 A schematic diagram showing a flow chart of a method for optimizing data management proposed in an embodiment of the present invention;

[0035] Figure 2 A structural diagram of a data management optimization device proposed in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0037] The embodiment of the present application provides a method for optimizing data management, which is applied to a database connected to a hardware accelerator, such as Figure 1 As shown, the method includes the following steps:

[0038] S101, defining a page attribute data structure, and initializing the acceleration card storage, host cache page information, the host page cache and acceleration card page management information according to the page attribute data structure, the capacity of the acceleration card storage and the capacity of the host page cache.

[0039] The page attribute data structure includes a database name, a data table number, a host side page number, an accelerator card side page number, a dirty page mark, a page location, and tuple information.

[0040] In this embodiment, the step of defining a page attribute data structure and initializing the accelerator card storage, host cache page information, the host page cache, and accelerator card page management information according to the page attribute data structure, the accelerator card storage capacity, and the host page cache capacity in S101 includes:

[0041] Initialize the accelerator card storage and host cache page information, load the data page from the host's persistent storage and store it in the accelerator card storage according to the page attribute data structure and the accelerator card storage capacity, generate the accelerator card-side page number, and update the host cache page information, including the database name, the data table number, the data page number, the dirty page flag, the page location, and the tuple information of the loaded data page;

[0042] Initializing the host page cache and the accelerator card page management information, and setting both the host page cache and the accelerator card page management information to empty according to the initialized accelerator card storage and the host cache page information;

[0043] The initialized dirty page flags are all null values. When the data page is modified to form a dirty page, the dirty page flag is set to a dirty page.

[0044] S102, receiving and sending an operation instruction to the accelerator, the accelerator processes the data page according to the type of the operation instruction and the page attribute data structure, marks the data page as a dirty page, generates dirty page data, and returns the processing result to the host.

[0045] In which, when the accelerator returns the processing result to the host, it also returns the dirty page modification information corresponding to the operation instruction. The host caches the dirty page data in the host page cache according to the dirty page modification information. When the dirty page data is written back to the host storage, the dirty page data is directly read from the host page cache.

[0046] The types of operation instructions include adding operation instructions, deleting operation instructions, modifying operation instructions, and querying operation instructions. The operation instructions are obtained by parsing and optimizing the user's database operation request or generated by the auxiliary process of the host. Among them, the operation instructions generated by the auxiliary process of the host do not include the query operation instructions.

[0047] To achieve accelerated processing of add, delete, and modify database operation instructions, in this embodiment, the step of receiving and sending an operation instruction to the accelerator in S102, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, generating dirty page data, and returning the processing result to the host includes the step of processing the add, delete, and modify operation request, specifically:

[0048] The host receives and sends the operation instruction to the accelerator;

[0049] When the type of the operation instruction received by the accelerator from the host is the add operation instruction, the delete operation instruction, the modify operation instruction, or the operation instruction generated by the auxiliary process of the host, then

[0050] The host sends a first execution plan obtained by parsing and optimizing the operation instruction to the accelerator. The accelerator processes the data page according to the first execution plan and the page attribute data structure, sets the dirty page flag of the processed data page to a dirty page to form the dirty page data, updates the host cache page information, and records the dirty page modification information, including the accelerator card side page number, the database name, the data table number, the page position, and the tuple information of the dirty page.

[0051] The accelerator returns the processing result to the host and returns the dirty page modification information to the host;

[0052] The host caches the dirty page data and the dirty page modification information according to the dirty page modification information and the host cache page information.

[0053] It is worth noting that in some actual specific application scenarios, the operation instructions generated by the auxiliary process of the host, such as the Vacuum instruction (reclaiming the storage space occupied by dead tuples), do not require parsing and optimization. The operation instructions generated by the auxiliary process of the host can be directly sent to the accelerator for execution, and the processing results are returned to the host after execution, and the dirty page modification information is returned to the host.

[0054] In order to reduce the consumption of communication resources during data interaction, in this embodiment, the host caches the dirty page data and the dirty page modification information according to the dirty page modification information and the host cache page information in the following steps:

[0055] If the host cache page information does not contain the accelerator card side page number of the dirty page, the dirty page modification information is stored in the host cache page information, and the host side page number is assigned, and the data page and metadata corresponding to the dirty page are read from the accelerator according to the dirty page modification information, and cached in the host page cache;

[0056] If the host cache page information contains the accelerator card side page number of the dirty page, the dirty page modification information is updated to the host cache page information, and according to the host side page number of the dirty page, the accelerator card side page number, the page position and the tuple information, the metadata corresponding to the dirty page is read from the accelerator and updated to the host page cache, and data offset processing is performed on the data page corresponding to the dirty page in the host page cache.

[0057] In order to facilitate those skilled in the art to better understand the technical solution provided by the present invention, this step will be further explained by taking database 1 as an example.

[0058] When processing the add operation instruction, the delete operation instruction, the modify operation instruction, or the operation instruction generated by the auxiliary process of the host, the accelerator returns a processing result and the dirty page modification information to the host, where the dirty page modification information includes the accelerator card side page number, the database name, the data table number, the page position, and the tuple information of the dirty page; wherein the database name is database1, the data table number is table1, and the accelerator card side page number is 00000001;

[0059] The host receives the processing result returned by the accelerator and the dirty page modification information, and after searching according to the accelerator card side page number, determines that the data page corresponding to the accelerator card side page number is returned for the first time, that is, the accelerator card side page number of the dirty page does not exist in the host cache page information, then the host side page number is assigned to the data page corresponding to the accelerator card side page number, the host side page number is 12, and a record as shown in Table 1 is added to the host cache page information. At the same time, according to the dirty page modification information, the entire data page and all metadata corresponding to the dirty page are read from the accelerator and cached in the host page cache.

[0060] Table 1

[0061] Database name / table number Accelerator card side page number Host side page number database1 / table1 00000001 12

[0062] When the accelerator returns the processing result and the dirty page modification information to the host again, after the host receives the processing result and the dirty page modification information returned by the accelerator, the host determines, based on the accelerator card side page number and the record of the host cache page information, that the dirty page with the accelerator card side page number already exists in the host cache page information. Wherein, the accelerator card side page number is 00000001, then there is no need to add a new record to the host cache page information. The host reads the metadata corresponding to the dirty page from the accelerator based on the host side page number, the accelerator card side page number, the page position, and the tuple information of the dirty page, updates it to the host page cache, and performs data offset processing on the data page with the host side page number 12 in the host page cache. Then, the host cache page information is refreshed without adding a new record, thereby achieving caching processing of the dirty page and the dirty page modification information according to the dirty page modification information and the host cache page information, reducing communication resource consumption during data interaction, and improving the processing efficiency of operation instructions.

[0063] To achieve accelerated processing of a database query operation instruction, the step of receiving and sending an operation instruction to the accelerator in S102, the accelerator processing a data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, generating dirty page data, and returning a processing result to the host includes the step of query operation request processing, specifically:

[0064] The host receives the query operation instruction of the user, and parses and optimizes the query operation instruction to obtain a second execution plan corresponding to the query operation instruction;

[0065] According to the second execution plan and the host cache page information, if a query result corresponding to the second execution plan exists in the host page cache, and an execution cost of the second execution plan on the host is less than or equal to an execution cost of the second execution plan on the accelerator, then executing the second execution plan on the host, and obtaining and returning the query result from the host page cache according to the second execution plan;

[0066] If the query result corresponding to the second execution plan does not exist in the host page cache, or the query result corresponding to the second execution plan exists in the host page cache but the execution cost of the second execution plan on the host is greater than the execution cost of the second execution plan on the accelerator, the second execution plan is sent to the accelerator, and the accelerator obtains and returns the query result from the acceleration card storage of the accelerator according to the second execution plan.

[0067] To ensure that the processing result of the returned operation instruction is more accurate and efficient, the step of receiving and sending the operation instruction to the accelerator in S102, wherein the accelerator processes the data page according to the type of the operation instruction and the page attribute data structure, further includes the step of processing a data page fault, specifically:

[0068] When the data page does not exist in the accelerator card storage,

[0069] The accelerator issues a page fault request to the host;

[0070] The host parses the operation instruction according to the page fault request to obtain the database name and the data table number, loads the data page corresponding to the operation instruction from the host's persistent storage according to the database name and the data table number, and allocates the accelerator card side page number;

[0071] Performing page replacement processing on the data page corresponding to the operation instruction according to the storage capacity of the acceleration card and the capacity of the host page cache.

[0072] To improve the accuracy and speed of data page replacement and reasonably utilize the cache space of the host and the accelerator card, the step of performing page replacement processing on the data page corresponding to the operation instruction according to the storage capacity of the accelerator card and the capacity of the host page cache includes host-side page replacement processing and accelerator-side page replacement processing;

[0073] in,

[0074] The steps of the host side replacement page processing are:

[0075] 1) When the available capacity of the host page cache is greater than or equal to the data size of the data page corresponding to the operation instruction,

[0076] The data page corresponding to the operation instruction is cached in the host page cache, and the host side page number is assigned to the data page corresponding to the operation instruction, the host side page number, the accelerator card side page number, the database name and the data table number are stored in the host cache page information, and the dirty page mark is initialized to a null value;

[0077] Sending a data page corresponding to the operation instruction to the accelerator;

[0078] When the available capacity of the host page cache is less than the data size of the data page corresponding to the operation instruction,

[0079] The host sends a data page corresponding to the operation instruction to the accelerator;

[0080] The steps of the accelerator side replacement page processing are:

[0081] The accelerator receives the data page corresponding to the operation instruction and the page number of the acceleration card side, and stores the data page corresponding to the operation instruction according to the available capacity of the acceleration card storage.

[0082] 1) When the available capacity of the accelerator card storage is greater than or equal to the data volume of the data page corresponding to the operation instruction,

[0083] Storing the data page corresponding to the operation instruction in the acceleration card storage;

[0084] The acceleration card side page number, the database name and the data table number are updated to the host cache page information respectively, and the dirty page mark is initialized to a null value.

[0085] 2) When the available capacity of the accelerator card storage is less than the data volume of the data page corresponding to the operation instruction,

[0086] Deleting the stored data page from the accelerator card storage according to a preset data page replacement algorithm and the amount of data in the data page corresponding to the operation instruction, and storing the data page corresponding to the operation instruction in the accelerator card storage;

[0087] The acceleration card side page number, the database name and the data table number are updated to the host cache page information respectively, and the dirty page mark is initialized to a null value.

[0088] The preset data page replacement algorithm may be set according to the residence time, access times, weight and data size of the data page.

[0089] For example, when the page data that needs to be cached exceeds the cache space threshold set by the acceleration card storage, the cache page replacement operation is performed according to a predetermined replacement strategy (such as the LRU page replacement algorithm to eliminate the page that has not been used the longest). When the accelerator generates a page swap request and performs a swap operation on the A table, the host reads the corresponding A table page from the persistent storage medium, and the host assigns the corresponding acceleration card cache number to it; if the cache space of the host page cache on the host is sufficient, the A table page data is cached at the same time, and the page data is sent to the accelerator cache through the interface, and the host cache page information on the host is also updated. If the cache space of the host page cache on the host is sufficient, but there are pages that meet the replacement strategy, the A table page data can also be replaced, the host cache page information is updated, and the page data is sent to the accelerator through the interface, thereby completing the replacement page processing of the host and the accelerator.

[0090] S103: The host writes the dirty page data back to the host storage according to the dirty page mark and the processing result.

[0091] In this embodiment, the step of writing the dirty page data back to the host storage according to the dirty page mark and the processing result in S103 includes:

[0092] The host stores the dirty page data in the host page cache to the persistent storage of the slave host according to the dirty page write-back policy and the dirty page flag of the data page, and sets the dirty page flag of the data page to a null value;

[0093] in,

[0094] The dirty page write-back policy is set according to the residence time, polling order, data size and modification times of the dirty page data in the host page cache.

[0095] For example, when the dirty page write-back time arrives on the host side, the host writes the dirty page data in the host page cache that complies with the policy back to the persistent storage medium according to the predetermined dirty page write-back strategy, and clears the corresponding dirty page mark. Among them, the predetermined dirty page write-back strategy can be preset as needed, including writing out dirty pages with longer residence time in the memory first, or polling writing, or writing out dirty pages with more changes first according to the amount of data modified in the dirty page, etc. It is worth noting that the dirty page can be marked as a flag bit as needed, such as a flag member variable in the page attribute data structure, which is set to 1 for dirty pages and 0 for normal. The page set to dirty page is also added to the dirty page linked list, and is removed when it is set to 0, thereby realizing the dirty page marking of the data page and completing efficient processing and management of the data page according to the dirty page mark.

[0096] Optionally, after writing the dirty page data back to the host storage according to the dirty page mark, the method further includes a step of accelerator failure recovery, specifically:

[0097] When the accelerator fails,

[0098] The host receives and processes the data page according to the type of the operation instruction and the page attribute data structure, adds a dirty page mark to the processed data page, and returns a processing result;

[0099] The host restores the dirty page data to the accelerator card storage of the accelerator based on the log information of the accelerator, and loads the data page from the host's persistent storage and stores it in the accelerator card storage based on the capacity of the accelerator card storage, generates the accelerator card-side page number, and updates the host cache page information on the accelerator;

[0100] After the accelerator returns to normal, it starts to receive the operation instruction sent by the host again.

[0101] For example, when the host detects that the accelerator fails, the host performs REDO / UNDO (redo / undo) operations through the saved log information. At this time, REDO / UNDO (redo / undo) operates the page data through the accelerator card page management module. The specific process is: based on the content of XLOG (log information stored on the host), starting from the most recent checkpoint (internal event) point, replay is performed, including writing back dirty pages, rolling back the data of uncommitted transactions to normal, and other operations. These page operations need to be taken over by the accelerator card page management module on the host, which determines whether the processing is in the operation cache or needs to be read from the persistent medium.

[0102] After the REDO / UNDO process is complete, all subsequent database operations are performed on the host side. The host receives and processes the data page based on the type of the operation instruction and the page attribute data structure, adds a dirty page mark to the processed data page, and returns the processing result. If the accelerator card fails, the accelerator card-side page number information stored in the host cache page information becomes invalid.

[0103] When the accelerator returns to normal, the host suspends service and starts to enter the data synchronization operation. The information saved by the accelerator card cache management module is obtained to obtain the current cached Table list, and the corresponding page data is loaded into the accelerator card storage according to the table information in the Table list, with priority loading of the page data in the cache, and then loading the page data of the tables in the remaining Table list in the persistent storage medium. When the cached tables are loaded, the remaining page data are loaded from the persistent storage medium according to the initial predetermined table loading strategy (for example, according to the order of table storage size, priority is given to loading tables with small storage capacity; or priority is given to loading tables with foreign keys to tables in the cache, or with high relevance to the content of tables in the cache), until the memory capacity set by the accelerator card is reached. Among them, the predetermined table loading strategy can be selected by the user according to the corresponding table loading strategy, including according to the order of table storage size, priority loading of tables with small storage capacity, or priority loading of tables with foreign keys to tables in the cache, or with high relevance to the content of tables in the cache, etc.

[0104] After the data synchronization operation is completed, subsequent database operations are switched back to the accelerator for execution.

[0105] By applying the above technical solution, when the accelerator returns the processing result to the host, it also returns the dirty page modification information corresponding to the operation instruction. The host caches the dirty page data modified according to the dirty page modification information in the host page cache. When the dirty page data is written back to the host storage, the dirty page data is directly read from the host page cache. This enables the accelerator card to return the page data corresponding to the processing result of the operation instruction at the same time as returning the processing result of the operation instruction, and completes the cache on the host in advance for the corresponding user's query request, data persistent storage and data recovery. On the basis of ensuring data cache consistency, the number of interface-based data interactions between the accelerator card and the host is greatly reduced, and technical problems such as frequent data interaction, data transmission delay and PCIe interface delay in the prior art are solved, thereby completing the acceleration of the database system processing performance based on the accelerator card, reducing the communication overhead caused by data interaction, and improving the data processing performance and reliability of the database system, as well as the applicability of the accelerator card.

[0106] Corresponding to a data management optimization method in an embodiment of the present invention, the present invention also discloses a data management optimization device, which is applied to a database system connected to an accelerator, such as Figure 2 As shown, the device includes a parsing and optimization module 201, an interface processing module 202, an accelerator card page management module 203, a page cache module 204, an accelerated execution module 205, an accelerator card storage module 206, and a host cache page information management module 207;

[0107] The parsing and optimization module, the interface processing module, the accelerator card page management module, and the page cache module are deployed in the host, and the acceleration execution module, the accelerator card storage module, and the host cache page information management module are deployed in the accelerator.

[0108] The parsing and optimization module 201 is used to receive a user's database operation request, and parse and optimize the user's database operation request to obtain an execution plan corresponding to the user's database operation request;

[0109] The interface processing module 202 is configured to send an execution plan corresponding to the user's database operation request to the accelerator;

[0110] The accelerator card page management module 203 is configured to cache the accelerator card side page number, the database name, the data table number, the data page number, the dirty page flag, the page location, and the tuple information corresponding to the dirty page data from the host cache page information management module when the accelerator processes the data page according to the type of the operation instruction and the page attribute data structure and returns the processing result, and generate and store the host side page number; and when performing the host side page replacement processing on the data page corresponding to the operation instruction according to the capacity of the host page cache, assign the host side page number to the data page corresponding to the operation instruction, and store the host side page number, the accelerator card side page number, the database name, the data table number, and the dirty page flag;

[0111] The page cache module 204 is configured to cache the dirty page data in the accelerator card storage module and, based on the capacity of the host page cache, cache the data page corresponding to the operation instruction when performing the host-side page replacement processing on the data page corresponding to the operation instruction;

[0112] The accelerated execution module 205 is configured to receive and execute the execution plan corresponding to the user's database operation request issued by the interface processing module and send it to the accelerator;

[0113] The accelerator card storage module 206 is configured to load and store the data page from the host's persistent storage according to the page attribute data structure and the accelerator card storage capacity, and generate the accelerator card-side page number;

[0114] The host cache page information management module 207 is used to store and update the host cache page information on the accelerator, including the database name, the data table number, the data page number, the dirty page mark, the page location and the tuple information of the loaded data page, and generate and store the accelerator card side page number.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for optimizing data management, applied to a database system connected to an accelerator, comprising: S101, defining a page attribute data structure, and initializing the accelerator card storage, host cache page information, the host page cache, and accelerator card page management information according to the page attribute data structure, the accelerator card storage capacity, and the host page cache capacity; S102, receiving and sending an operation instruction to the accelerator, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, generating dirty page data, and returning the processing result to the host; S103, the host writes the dirty page data back to the host storage according to the dirty page mark and the processing result; The accelerator is characterized in that, when returning a processing result to the host, it also returns dirty page modification information corresponding to the operation instruction; the host caches the dirty page data in the host page cache according to the dirty page modification information; when writing the dirty page data back to the host storage, the dirty page data is directly read from the host page cache; in, The page attribute data structure includes database name, data table number, host side page number, accelerator card side page number, dirty page mark, page location and tuple information; The types of operation instructions include adding operation instructions, deleting operation instructions, modifying operation instructions, and querying operation instructions. The operation instructions are obtained by parsing and optimizing the user's database operation request or generated by the auxiliary process of the host. Among them, the operation instructions generated by the auxiliary process of the host do not include the query operation instructions.

2. The method according to claim 1, wherein The step of defining a page attribute data structure and initializing the accelerator card storage, host cache page information, the host page cache, and accelerator card page management information according to the page attribute data structure, the accelerator card storage capacity, and the host page cache capacity in S101 includes: Initialize the accelerator card storage and host cache page information, load the data page from the host's persistent storage and store it in the accelerator card storage according to the page attribute data structure and the accelerator card storage capacity, generate the accelerator card-side page number, and update the host cache page information, including the database name, the data table number, the data page number, the dirty page flag, the page location, and the tuple information of the loaded data page; Initializing the host page cache and the accelerator card page management information, and setting both the host page cache and the accelerator card page management information to empty according to the initialized accelerator card storage and the host cache page information; The initialized dirty page flags are all null values. When the data page is modified to form a dirty page, the dirty page flag is set to a dirty page.

3. The method according to claim 1, wherein The step S102 of receiving and sending an operation instruction to the accelerator, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, generating dirty page data, and returning the processing result to the host includes the steps of adding an operation request process, deleting an operation request process, and modifying an operation request process, specifically: The host receives and sends the operation instruction to the accelerator; When the type of the operation instruction received by the accelerator from the host is the add operation instruction, the delete operation instruction, the modify operation instruction, or the operation instruction generated by the auxiliary process of the host, then The host sends a first execution plan obtained by parsing and optimizing the operation instruction to the accelerator. The accelerator processes the data page according to the first execution plan and the page attribute data structure, sets the dirty page flag of the processed data page to a dirty page to form the dirty page data, updates the host cache page information, and records the dirty page modification information, including the accelerator card side page number, the database name, the data table number, the page position, and the tuple information of the dirty page. The accelerator returns the processing result to the host and returns the dirty page modification information to the host; The host caches the dirty page data and the dirty page modification information according to the dirty page modification information and the host cache page information.

4. The method according to claim 3, wherein The host performs cache processing on the dirty page data and the dirty page modification information according to the dirty page modification information and the host cache page information, comprising: If the host cache page information does not contain the accelerator card side page number of the dirty page, the dirty page modification information is stored in the host cache page information, and the host side page number is assigned, and the data page and metadata corresponding to the dirty page are read from the accelerator according to the dirty page modification information, and cached in the host page cache; If the host cache page information contains the accelerator card side page number of the dirty page, the dirty page modification information is updated to the host cache page information, and according to the host side page number of the dirty page, the accelerator card side page number, the page position and the tuple information, the metadata corresponding to the dirty page is read from the accelerator and updated to the host page cache, and data offset processing is performed on the data page corresponding to the dirty page in the host page cache.

5. The method according to claim 1, wherein The step of receiving and sending an operation instruction to the accelerator, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, generating dirty page data, and returning the processing result to the host includes the step of querying the operation request processing, specifically: The host receives the query operation instruction of the user, and parses and optimizes the query operation instruction to obtain a second execution plan corresponding to the query operation instruction; According to the second execution plan and the host cache page information, if a query result corresponding to the second execution plan exists in the host page cache, and an execution cost of the second execution plan on the host is less than or equal to an execution cost of the second execution plan on the accelerator, then executing the second execution plan on the host, and obtaining and returning the query result from the host page cache according to the second execution plan; If the query result corresponding to the second execution plan does not exist in the host page cache, or the query result corresponding to the second execution plan exists in the host page cache but the execution cost of the second execution plan on the host is greater than the execution cost of the second execution plan on the accelerator, the second execution plan is sent to the accelerator, and the accelerator obtains and returns the query result from the acceleration card storage of the accelerator according to the second execution plan.

6. The method according to claim 1, wherein The step of receiving and sending an operation instruction to the accelerator, wherein the accelerator processes the data page according to the type of the operation instruction and the page attribute data structure, further includes the step of processing a data page fault, specifically: When the data page does not exist in the accelerator card storage, The accelerator issues a page fault request to the host; The host parses the operation instruction according to the page fault request to obtain the database name and the data table number, loads the data page corresponding to the operation instruction from the host's persistent storage according to the database name and the data table number, and allocates the accelerator card side page number; Performing page replacement processing on the data page corresponding to the operation instruction according to the storage capacity of the acceleration card and the capacity of the host page cache.

7. The method according to claim 6, wherein The step of performing page replacement processing on the data page corresponding to the operation instruction according to the storage capacity of the accelerator card and the capacity of the host page cache includes host-side page replacement processing and accelerator-side page replacement processing; in, The steps of the host side replacement page processing are: When the available capacity of the host page cache is greater than or equal to the data size of the data page corresponding to the operation instruction, caching the data page corresponding to the operation instruction in the host page cache, assigning the host-side page number to the data page corresponding to the operation instruction, storing the host-side page number, the accelerator card-side page number, the database name, and the data table number in the host cache page information, and initializing the dirty page mark to a null value; Sending a data page corresponding to the operation instruction to the accelerator; When the available capacity of the host page cache is less than the data size of the data page corresponding to the operation instruction, The host sends a data page corresponding to the operation instruction to the accelerator; The steps of the accelerator side replacement page processing are: The accelerator receives the data page corresponding to the operation instruction and the page number of the acceleration card side, and stores the data page corresponding to the operation instruction according to the available capacity of the acceleration card storage. When the available capacity of the acceleration card storage is greater than or equal to the data volume of the data page corresponding to the operation instruction, Storing the data page corresponding to the operation instruction in the accelerator card storage; Update the acceleration card side page number, the database name and the data table number to the host cache page information respectively, and initialize the dirty page mark to a null value; When the available capacity of the acceleration card storage is less than the data volume of the data page corresponding to the operation instruction, Deleting the stored data page from the accelerator card storage according to a preset data page replacement algorithm and the amount of data in the data page corresponding to the operation instruction, and storing the data page corresponding to the operation instruction in the accelerator card storage; Update the acceleration card side page number, the database name and the data table number to the host cache page information respectively, and initialize the dirty page mark to a null value; The preset data page replacement algorithm may be set according to the residence time, access times, weight and data size of the data page.

8. The method according to claim 1, wherein The step S103 of the host writing the dirty page data back to the host storage according to the dirty page mark and the processing result includes: The host stores the dirty page data in the host page cache to the persistent storage of the slave host according to the dirty page write-back policy and the dirty page flag of the data page, and sets the dirty page flag of the data page to a null value; in, The dirty page write-back policy is set according to the residence time, polling order, data size and modification times of the dirty page data in the host page cache.

9. The method according to claim 1, wherein After writing the dirty page data back to the host storage according to the dirty page mark, the step of accelerator failure recovery is further included, specifically: When the accelerator fails, The host receives and processes the data page according to the type of the operation instruction and the page attribute data structure, adds a dirty page mark to the processed data page, and returns a processing result; The host restores the dirty page data to the accelerator card storage of the accelerator based on the log information of the accelerator, and loads the data page from the host's persistent storage and stores it in the accelerator card storage based on the capacity of the accelerator card storage, generates the accelerator card-side page number, and updates the host cache page information on the accelerator; After the accelerator returns to normal, it starts to receive the operation instruction sent by the host again.

10. A device for implementing the data management optimization method of claim 1, applied to a database system connected to an accelerator, characterized in that: The device includes a parsing and optimization module, an interface processing module, an accelerator card page management module, a page cache module, an accelerated execution module, an accelerator card storage module, and a host cache page information management module; in, The parsing optimization module, the interface processing module, the accelerator card page management module, and the page cache module are deployed in the host, and the acceleration execution module, the accelerator card storage module, and the host cache page information management module are deployed in the accelerator; The parsing and optimization module is used to receive a user's database operation request, and parse and optimize the user's database operation request to obtain an execution plan corresponding to the user's database operation request; The interface processing module is configured to send an execution plan corresponding to the user's database operation request to the accelerator; The accelerator card page management module is configured to cache the accelerator card side page number, the database name, the data table number, the data page number, the dirty page flag, the in-page position, and the tuple information corresponding to the dirty page data from the host cache page information management module when the accelerator processes the data page according to the type of the operation instruction and the page attribute data structure and returns the processing result, and generate and store the host side page number; and when performing host side page replacement processing on the data page corresponding to the operation instruction according to the capacity of the host page cache, assign the host side page number to the data page corresponding to the operation instruction, and store the host side page number, the accelerator card side page number, the database name, the data table number, and the dirty page flag; The page cache module is configured to cache the dirty page data in the accelerator card storage module and, based on the capacity of the host page cache, cache the data page corresponding to the operation instruction when performing host-side page replacement processing on the data page corresponding to the operation instruction; The acceleration execution module is configured to receive and execute the execution plan corresponding to the user's database operation request issued by the interface processing module and send it to the accelerator; The accelerator card storage module is configured to load and store the data page from the host's persistent storage according to the page attribute data structure and the accelerator card storage capacity, and generate the accelerator card-side page number; The host cache page information management module is used to store and update the host cache page information on the accelerator, including the database name, the data table number, the data page number, the dirty page mark, the page position and the tuple information of the loaded data page, and generate and store the accelerator card side page number.

Images