Metadata caching method, system, device and equipment in database and medium

Abstract

The application relates to the field of electronic technology and discloses a metadata caching method, system, device, equipment and medium in a database. The method comprises the following steps: obtaining first metadata of a first process and second metadata of a second process in a target database, wherein the first metadata comprises first system directory data, and the second metadata comprises second system directory data; storing at least one target tuple data existing in the first system directory data of the first process and the second system directory data of the second process into a first shared resource pool; and the first process and the second process only need to store address information of the target tuple data in the first shared resource pool in corresponding local caches, and do not need to store the target tuple data in respective local caches, so that the memory occupation of metadata caching in the database is effectively reduced, and the problem of excessive memory occupation caused by redundant metadata copies cached by each process in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of electronic technology, in particular to a metadata caching method, system, device and equipment in a database and a medium. BACKGROUND

[0002] System table metadata is the core descriptive information stored by system tables in a database, covering the definitions and association relationships of database, table, column, index, constraint, and permission objects. To improve the access efficiency of system table metadata, a relational database system (PostgreSQL) introduces multiple cached data in each process: system catalog cache (CatCache / SysCache) and relation descriptor cache (RelCache). PostgreSQL adopts a process architecture model, and each client connection starts a backend process, and each backend process has an independent metadata cache. Since different backend processes have their respective RelCache and CatCache, when the number of connections is large or the number of objects in the database (tables, views, indexes, etc.) is large, the database instance will consume a large amount of memory. When the number of connections N is large and the metadata cache M is large, the PostgreSQL cluster will occupy a lot of memory (N*M), and in severe cases, it will cause out of memory (OOM). SUMMARY

[0003] Therefore, the present application provides a metadata caching method, system, device and equipment in a database and a medium to solve the problem of excessive memory occupation caused by the redundant caching of metadata copies in each process under the PostgreSQL multi-process architecture model in the related art.

[0004] In a first aspect, the present application provides a metadata caching method in a database, the method comprising: obtaining first metadata of a first process and second metadata of a second process in a target database, the first metadata comprising first system catalog data, and the second metadata comprising second system catalog data; determining at least one target tuple data existing in both the first system catalog data and the second system catalog data; determining first target system catalog data of the first process based on the first system catalog data and the target tuple data, the first target system catalog data not containing the target tuple data; determining second target system catalog data of the second process based on the second system catalog data and the target tuple data, the second target system catalog data not containing the target tuple data; storing the target tuple data into a first shared resource pool constructed in advance, and determining first address information of the target tuple data in the first shared resource pool; storing the first address information and the first target system catalog data into a local cache of the first process, and storing the first address information and the second target system catalog data into a local cache of the second process.

[0005] The metadata caching method in a database provided by the present application comprises: obtaining first metadata of a first process and second metadata of a second process in a target database, the first metadata comprising first system catalog data, and the second metadata comprising second system catalog data; determining at least one target tuple data existing in both the first system catalog data and the second system catalog data; determining first target system catalog data of the first process based on the first system catalog data and the target tuple data, the first target system catalog data not containing the target tuple data; determining second target system catalog data of the second process based on the second system catalog data and the target tuple data, the second target system catalog data not containing the target tuple data; storing the target tuple data into a first shared resource pool constructed in advance, and determining first address information of the target tuple data in the first shared resource pool; storing the first address information and the first target system catalog data into a local cache of the first process, and storing the first address information and the second target system catalog data into a local cache of the second process. The method provided by the present application stores at least one target tuple data existing in both the first system catalog data of the first process and the second system catalog data of the second process into a first shared resource pool, and the first process and the second process only need to store address information of the target tuple data in the first shared resource pool in corresponding local caches, without storing the target tuple data in respective local caches, thereby effectively reducing memory occupation of metadata caching in a database, and solving the problem of excessive memory occupation caused by redundant metadata copies cached by respective processes in the prior art.

[0006] In an optional embodiment, the first metadata further comprises first relationship descriptor data, and the second metadata further comprises second relationship descriptor data, and the method further comprises: determining at least one target tuple descriptor data existing in both the first relationship descriptor data and the second relationship descriptor data; determining first target relationship descriptor data of the first process based on the first relationship descriptor data and the target tuple descriptor data, the first target relationship descriptor data not containing the target tuple descriptor data; determining second target relationship descriptor data of the second process based on the second relationship descriptor data and the target tuple descriptor data, the second target relationship descriptor data not containing the target tuple descriptor data; storing the target tuple descriptor data into the pre-constructed second shared resource pool, and determining second address information of the target tuple descriptor data in the second shared resource pool; storing the second address information and the first target relationship descriptor data into the local cache of the first process, and storing the second address information and the second target relationship descriptor data into the local cache of the second process.

[0007] The method provided by the optional embodiment stores at least one target tuple descriptor data existing in both the first relationship descriptor data of the first process and the second relationship descriptor data of the second process into the second shared resource pool, and the first process and the second process only need to store address information of the target tuple descriptor data in the second shared resource pool in the corresponding local cache, and do not need to store the target tuple descriptor data in the respective local cache, thereby effectively relieving the pressure of the memory and solving the problem of excessive memory occupation caused by redundant metadata copies in the respective processes in the prior art.

[0008] In an optional embodiment, the method further comprises: when a query request for target tuple data of the first process or the second process is received, retrieving first address information of the target tuple data from the local cache of the corresponding process, the target tuple data being tuple data existing in both the first system directory data of the first process and the second system directory data of the second process; and querying the target tuple data in the first shared resource pool based on the first address information.

[0009] In an optional embodiment, the method further comprises: when a query request for target tuple descriptor data of the first process or the second process is received, retrieving second address information of the target tuple descriptor data from the local cache of the corresponding process, the target tuple data being tuple data existing in both the first system directory data of the first process and the second system directory data of the second process; and querying the target tuple descriptor data in the second shared resource pool based on the second address information.

[0010] In a second aspect, the present application provides a metadata caching system in a database, comprising: a local cache of a first process, a local cache of a second process, a first shared resource pool, and a cache management module; the cache management module is configured to execute the metadata caching method in a database according to the first aspect or any one of the corresponding embodiments.

[0011] The metadata caching system in a database provided by the present application comprises: a local cache of a first process, a local cache of a second process, a first shared resource pool, and a cache management module; the cache management module stores at least one target tuple data existing in the first system directory data of the first process and the second system directory data of the second process into the first shared resource pool, and the first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in the corresponding local caches, and do not need to store the target tuple data in the respective local caches, thereby effectively reducing the memory occupation of the metadata caching in the database, and solving the problem of excessive memory occupation caused by the redundant metadata copies cached by each process in the prior art.

[0012] In a third aspect, the present application provides a metadata caching device in a database, comprising: an acquisition module configured to acquire first metadata of a first process and second metadata of a second process in a target database, wherein the first metadata comprises first system directory data, and the second metadata comprises second system directory data; a first determination module configured to determine at least one target tuple data existing in the first system directory data and the second system directory data; a second determination module configured to determine first target system directory data of the first process based on the first system directory data and the target tuple data, wherein the first target system directory data does not contain the target tuple data; a third determination module configured to determine second target system directory data of the second process based on the second system directory data and the target tuple data, wherein the second target system directory data does not contain the target tuple data; a first construction module configured to store the target tuple data into a first shared resource pool constructed in advance, and determine first address information of the target tuple data in the first shared resource pool; and a first storage module configured to store the first address information and the first target system directory data into a local cache of the first process, and store the first address information and the second target system directory data into a local cache of the second process.

[0013] In some alternative embodiments, the apparatus further comprises: the first metadata further comprises first relationship descriptor data, and the second metadata further comprises second relationship descriptor data; the apparatus further comprises: a fourth determining module configured to determine at least one target tuple descriptor data existing in both the first relationship descriptor data and the second relationship descriptor data; a fifth determining module configured to determine first target relationship descriptor data of the first process based on the first relationship descriptor data and the target tuple descriptor data, the first target relationship descriptor data not containing the target tuple descriptor data; a sixth determining module configured to determine second target relationship descriptor data of the second process based on the second relationship descriptor data and the target tuple descriptor data, the second target relationship descriptor data not containing the target tuple descriptor data; a second constructing module configured to store the target tuple descriptor data into the second shared resource pool constructed in advance, and determine second address information of the target tuple descriptor data in the second shared resource pool; and a second storing module configured to store the second address information and the first target relationship descriptor data into the local cache of the first process, and store the second address information and the second target relationship descriptor data into the local cache of the second process.

[0014] In a fourth aspect, the present application provides a computer device, comprising: a memory and a processor, which are connected with each other in communication, and the memory stores computer instructions; the processor executes the computer instructions to perform the metadata caching method in the database according to the first aspect or any one of the corresponding embodiments thereof, or perform the metadata querying method in the database according to the second aspect or any one of the corresponding embodiments thereof.

[0015] In a fifth aspect, the present application provides a computer readable storage medium, which stores computer instructions for making a computer execute the metadata caching method in the database according to the first aspect or any one of the corresponding embodiments thereof.

[0016] In a sixth aspect, the present application provides a computer program product, which comprises computer instructions for making a computer execute the metadata caching method in the database according to the first aspect or any one of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0018] Figure 1is a flowchart of a metadata caching method in a database according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of another metadata caching method in a database according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a global system cache architecture according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a global relation cache architecture according to an embodiment of the present application;

[0022] Figure 5 is a structural block diagram of a metadata caching system in a database according to an embodiment of the present application

[0023] Figure 6 is a structural block diagram of a metadata caching apparatus in a database according to an embodiment of the present application;

[0024] Figure 7 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0026] In the related art, PostgreSQL adopts a process architecture model, and each client connection starts a backend process, and each backend process has an independent metadata cache. Since different backend processes have corresponding RelCache and CatCache, when the number of connections is large or the number of objects in the database (tables, views, indexes, etc.) is large, the database instance will consume a large amount of memory. When the number of connections N is large and the metadata cache M is large, the PostgreSQL cluster will occupy a lot of memory (N*M), and in severe cases, it will cause out of memory (OOM).

[0027] Therefore, the database metadata caching method provided by the embodiments of the present application can be applied to a server to realize caching of metadata in a database. The method provided by the embodiments of the present application stores at least one target tuple data existing in the first system catalog data of the first process and the second system catalog data of the second process in a first shared resource pool, and the first process and the second process only need to store address information of the target tuple data in the first shared resource pool in the corresponding local cache, and do not need to store the target tuple data in the respective local caches, thereby effectively reducing the memory occupation of metadata caching in the database and solving the problem of excessive memory occupation caused by redundant caching of metadata copies of each process in the prior art.

[0028] According to the embodiments of the present application, a database metadata caching method is provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] In the embodiments, a database metadata caching method is provided, which can be used in the server described above, Figure 1 is a flowchart of the database metadata caching method according to the embodiments of the present application, as Figure 1 shown, the flowchart includes the following steps:

[0030] In step S101, first metadata of a first process and second metadata of a second process in a target database are obtained, the first metadata includes first system catalog data, and the second metadata includes second system catalog data.

[0031] Exemplarily, the first process and the second process can be application program processes running on a server side (backend) and responsible for processing business logic, data interaction, resource scheduling and other core functions. In the embodiments of the present application, the first process and the second process are different backend (Backend) processes. System catalog (System Catalog, SysCache) data refers to a core component for storing metadata in a database caching system (Database Management System, DBMS), which is used to record database structure, object definition and associated information.

[0032] In step S102, at least one target tuple data existing in the first system catalog data and the second system catalog data is determined.

[0033] Exemplarily, the first system catalog data and the second system catalog data both contain a plurality of tuple data. In a relational database (such as PostgreSQL, MySQL), a tuple is a basic data unit of a relational model, corresponding to "a row of data" in a table. In the embodiment of the application, because the data structure of the SysCache is complex and various pointers are interpointed (for example, the CatCTup has pointers to the CatCache and the CatCList, the CatCTup is one of the core data structures of the catalog cache, and the design purpose is to efficiently manage the cache entries of metadata, and the CatCList is used to store the result list of a partial match query), the contents in these data structures are frequently updated. If the SysCache is shared as a whole, it is necessary to be locked to prevent concurrent operation. Not to mention the difficulty of implementation, because the SysCache is a high-frequency access module, the performance decline caused by lock competition is unacceptable. Therefore, the scheme of sharing the SysCache as a whole is not adopted. After analysis, the data pointed to by the tuple field is the largest part of the memory occupation in the SysCache, and the data will not change under a specific key and a specific version. Therefore, the tuple data is placed in the shared memory, and the local process pointer points to the tuple data, and different processes share the same tuple data. Therefore, the embodiment of the application needs to determine the target tuple data that exists in the first system catalog data and the second system catalog data at the same time.

[0034] In step S103, first target system catalog data of the first process is determined based on the first system catalog data and the target tuple data, and the first target system catalog data does not contain the target tuple data.

[0035] Exemplarily, in the embodiment of the application, other data in the first system catalog data except the target tuple data is taken as the first target system catalog data.

[0036] In step S104, second target system catalog data of the second process is determined based on the second system catalog data and the target tuple data, and the second target system catalog data does not contain the target tuple data.

[0037] Exemplarily, in the embodiment of the application, other data in the second system catalog data except the target tuple data is taken as the second target system catalog data.

[0038] In step S105, the target tuple data is stored in the first shared resource pool constructed in advance, and first address information of the target tuple data in the first shared resource pool is determined.

[0039] Exemplarily, the first shared resource pool is a storage area pre-constructed for storing tuple data shared by different processes, and in the embodiment of the present application, the first address information can include but is not limited to a tuple pointer. The tuple pointer (TuplePointer) is an identifier for locating the physical or logical address of a certain tuple (Tuple, i.e., a row record in a table) in a storage layer. It is a core mechanism for implementing efficient data access of a database and is commonly used in the storage engine and index design of a relational database.

[0040] In step S106, the first address information and the first target system directory data are stored into the local cache of the first process, and the first address information and the second target system directory data are stored into the local cache of the second process.

[0041] The metadata caching method provided in the embodiment stores at least one target tuple data existing in the first system directory data of the first process and the second system directory data of the second process into the first shared resource pool, and the first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in the corresponding local cache, without storing the target tuple data in the respective local caches, thereby effectively reducing the memory occupation of metadata caching in the database and solving the problem of excessive memory occupation caused by redundant caching of metadata copies by each process in the prior art.

[0042] In the embodiment, a metadata caching method in a database is provided, which can be used in the server described above, Figure 2 is a flowchart of the metadata caching method in a database according to the embodiment of the present application, as shown in the figure, the flowchart includes the following steps: Figure 2

[0043] In step S201, first metadata of a first process and second metadata of a second process in a target database are acquired, the first metadata includes first system directory data, and the second metadata includes second system directory data. For details, refer to step S101 of the embodiment shown in Figure 1 not repeated here.

[0044] In step S202, at least one target tuple data existing in the first system directory data and the second system directory data is determined. For details, refer to step S102 of the embodiment shown in Figure 1 not repeated here.

[0045] In step S203, first target system directory data of the first process is determined based on the first system directory data and the target tuple data, and the first target system directory data does not contain the target tuple data. For details, refer to step S103 of the embodiment shown in Figure 1 not repeated here.​

[0046] In step S204, the second target system catalog data of the second process is determined based on the second system catalog data and the target tuple data, and the second target system catalog data does not contain the target tuple data. For details, please refer to Figure 1 In step S104 of the embodiment shown in the figure, no further description is given here.

[0047] In step S205, the target tuple data is stored in the first shared resource pool constructed in advance, and the first address information of the target tuple data in the first shared resource pool is determined. For details, please refer to Figure 1 In step S105 of the embodiment shown in the figure, no further description is given here.

[0048] In step S206, the first address information and the first target system catalog data are stored in the local cache of the first process, and the first address information and the second target system catalog data are stored in the local cache of the second process. For details, please refer to Figure 1 In step S106 of the embodiment shown in the figure, no further description is given here.

[0049] In step S207, at least one target tuple descriptor data existing in the first relation descriptor data and the second relation descriptor data at the same time is determined.

[0050] Exemplarily, the relation descriptor data (Relation Descriptor Data) contains a plurality of element descriptor (Tuple Descriptor, TupleDesc) data, and the TupleDesc is a structure body mainly used for describing the information of each attribute (column) in the tuple, including the name, data type, length, etc. of the attribute. In the embodiment of the present application, like the SysCache, the Hash table (HashTable) of the entire RelCache is not selected to be shared. After analysis, the data pointed to by the TupleDesc field is the largest part of the memory in the RelCache, and under a specific version, the data will not change. Therefore, the TupleDesc data is put into the shared memory, and the local process pointer points to the TupleDesc data, and the same TupleDesc data of different processes is shared. Therefore, in the embodiment of the present application, the target tuple descriptor data existing in the relation descriptor data of different processes at the same time needs to be determined.

[0051] In step S208, the first target relation descriptor data of the first process is determined based on the first relation descriptor data and the target tuple descriptor data, and the first target relation descriptor data does not contain the target tuple descriptor data.

[0052] Exemplarily, in the embodiments of the present application, the data in the first relationship descriptor data other than the target tuple descriptor data is taken as the first target relationship descriptor data.

[0053] In step S209, the second target relationship descriptor data of the second process is determined based on the second relationship descriptor data and the target tuple descriptor data, and the second target relationship descriptor data does not contain the target tuple descriptor data.

[0054] Exemplarily, in the embodiments of the present application, the data in the second relationship descriptor data other than the target tuple descriptor data is taken as the second target relationship descriptor data.

[0055] In step S2010, the target tuple descriptor data is stored in the second shared resource pool constructed in advance, and the second address information of the target tuple descriptor data in the second shared resource pool is determined.

[0056] Exemplarily, the second shared resource pool is a storage area constructed in advance for storing the target tuple descriptor data, and in the embodiments of the present application, the second address information can include but is not limited to a TupleDesc pointer. In a database system, the TupleDesc pointer plays a key role, which points to a TupleDesc structure, and this structure is used to describe the structure information of a tuple (a row in a table).

[0057] In step S2011, the second address information and the first target relationship descriptor data are stored in the local cache of the first process, and the second address information and the second target relationship descriptor data are stored in the local cache of the second process.

[0058] In some optional embodiments, the above method further includes:

[0059] In step a1, when a query request for the target tuple data of the first process or the second process is received, the first address information of the target tuple data is retrieved from the local cache of the corresponding process, and the target tuple data is the tuple data existing in the first system directory data of the first process and the second system directory data of the second process at the same time.

[0060] Exemplarily, in the embodiments of the present application, the query request for the target tuple data can contain the unique identification information of the target tuple data to be queried, and the tuple pointer of the target tuple data is retrieved based on the unique identification information.

[0061] In step a2, the target tuple data is queried in the first shared resource pool based on the first address information.

[0062] Exemplarily, in the embodiments of the present application, the target tuple data is queried in the first shared resource pool based on the tuple pointer of the target tuple data.

[0063] Step a3, when receiving the query request of the target tuple descriptor data of the first process or the second process, retrieving the second address information of the target tuple descriptor data from the local cache of the corresponding process, the target tuple data being the tuple data existing in the first system directory data of the first process and the second system directory data of the second process at the same time.

[0064] Exemplarily, in the embodiment of the application, the query request of the target tuple descriptor data can contain the unique identification information of the target tuple descriptor data to be queried, and the TupleDesc pointer of the target tuple descriptor data is retrieved based on the unique identification information.

[0065] Step a4, querying the target tuple descriptor data in the second shared resource pool based on the second address information.

[0066] Exemplarily, in the embodiment of the application, the target tuple descriptor data is queried in the second shared resource pool based on the TupleDesc pointer of the target tuple descriptor data.

[0067] The metadata caching method in a database provided by the application will be described in detail below through a specific embodiment.

[0068] Embodiment:

[0069] (1) Global system cache (Global SysCache) architecture design: the data structure relationship of the whole SysCache system is complex, various pointers are interlinked (for example, CatCTup has pointers to CatCache and catclist) (CatCList is used to store the result list of partial match query), and the contents in these data structures are frequently updated. If the whole SysCache is shared, it must be locked to prevent concurrent operation. Not to mention the difficulty of implementation, due to the high-frequency access module of SysCache, the performance decline caused by lock competition is unacceptable. Therefore, the whole shared SysCache scheme is not adopted. After analysis, the data pointed to by the tuple field is the largest part of the memory occupation in SysCache, and the data will not change under a specific key and a specific version. Therefore, the tuple data is placed in the shared memory, and the local process pointer points to the tuple data. Different processes share the same tuple data. For two processes A and B, the schematic diagram of the global system cache architecture is shown in Figure 3 . HashTable is constructed in the shared memory, and different processes put the same tuple data into the shared memory, and the local process points to it through the pointer. The key of HashTable is the hash value of the memory data.

[0070] (2) Global RelCache architecture design: like SysCache, the HashTable of the entire RelCache is not selected to be shared.

[0071] After analysis, the data pointed by the TupleDesc field is the largest part of the memory occupation in the RelCache, and the data will not change in a specific version. Therefore, the TupleDesc data is put into the shared memory, and the local process pointer points to the TupleDesc data, and the same TupleDesc data of different processes is shared. The schematic diagram of the global RelCache architecture is shown as Figure 4 .

[0072] The HashTable is constructed in the shared memory, and the same TupleDesc data of different processes is put into the shared memory, and the local process points to the TupleDesc data through a pointer. The key of the HashTable is the hash value of the memory data.

[0073] The embodiment of the application further provides a metadata caching system in a database, as shown in Figure 5 , the system comprises a local cache 501 of a first process, a local cache 502 of a second process, a first shared resource pool 503, and a cache management module 504; the cache management module 504 is used for executing the metadata caching method in the database of the above-mentioned embodiment. Exemplarily, refer to the description of the related content in the above-mentioned embodiment for details, which will not be described here.

[0074] The metadata caching system in the database provided by the embodiment of the application comprises a local cache of a first process, a local cache of a second process, a first shared resource pool, and a cache management module; the cache management module stores at least one target tuple data in the first system directory data of the first process and the second system directory data of the second process into the first shared resource pool, and the first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in the corresponding local cache, and do not need to store the target tuple data in the respective local cache, thereby effectively reducing the memory occupation of the metadata caching in the database, and solving the problem of excessive memory occupation caused by the redundant caching of metadata copies of each process in the prior art.

[0075] In some optional embodiments, the system further comprises a shared memory base, a hash table of the shared memory base manages shared data of a shared resource pool, and provides a lookup, insertion, and deletion interface of the shared resource pool. The shared resource contains a reference count: the reference count is initialized to 1 when the backend process inserts the shared resource, the reference count is increased by 1 when the backend process references the shared resource, the reference count is decreased by 1 when the backend process releases the shared resource, and the shared resource is deleted if the reference count is reduced to 0. The shared resource pool includes 64 hash tables, and each hash table has a read-write lock: a write lock is required for modification (insertion and deletion) of the hash table, and a read lock is required for lookup of the hash table.

[0076] In the embodiment, a metadata caching apparatus in a database is also provided, which is used to implement the above-described embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0077] The embodiment provides a metadata caching apparatus in a database, as shown in the accompanying drawings, comprising: Figure 6

[0078] The obtaining module 601 is configured to obtain first metadata of a first process and second metadata of a second process in a target database, the first metadata comprising first system catalog data, and the second metadata comprising second system catalog data.

[0079] The first determining module 602 is configured to determine at least one target tuple data that exists in both the first system catalog data and the second system catalog data.

[0080] The second determining module 603 is configured to determine, based on the first system catalog data and the target tuple data, first target system catalog data of the first process, the first target system catalog data not containing the target tuple data.

[0081] The third determining module 604 is configured to determine, based on the second system catalog data and the target tuple data, second target system catalog data of the second process, the second target system catalog data not containing the target tuple data.

[0082] The first constructing module 605 is configured to store the target tuple data into a first shared resource pool constructed in advance, and determine first address information of the target tuple data in the first shared resource pool.

[0083] ​The first storage module 606 is configured to store the first address information and the first target system directory data into the local cache of the first process, and store the first address information and the second target system directory data into the local cache of the second process.

[0084] In some optional embodiments, the first metadata further comprises first relationship descriptor data, and the second metadata further comprises second relationship descriptor data, and the apparatus further comprises:

[0085] The fourth determination module is configured to determine at least one target tuple descriptor data existing in both the first relationship descriptor data and the second relationship descriptor data.

[0086] The fifth determination module is configured to determine, based on the first relationship descriptor data and the target tuple descriptor data, first target relationship descriptor data of the first process, and the first target relationship descriptor data does not contain the target tuple descriptor data.

[0087] The sixth determination module is configured to determine, based on the second relationship descriptor data and the target tuple descriptor data, second target relationship descriptor data of the second process, and the second target relationship descriptor data does not contain the target tuple descriptor data.

[0088] The second construction module is configured to store the target tuple descriptor data into the second shared resource pool constructed in advance, and determine second address information of the target tuple descriptor data in the second shared resource pool.

[0089] The second storage module is configured to store the second address information and the first target relationship descriptor data into the local cache of the first process, and store the second address information and the second target relationship descriptor data into the local cache of the second process.

[0090] In some optional embodiments, the apparatus further comprises:

[0091] The first retrieval module is configured to, when a query request for target tuple data of the first process or the second process is received, retrieve first address information of the target tuple data from the local cache of the corresponding process, the target tuple data being tuple data existing in both the first system directory data of the first process and the second system directory data of the second process.

[0092] The first query module is configured to query the target tuple data in the first shared resource pool based on the first address information.

[0093] In some optional embodiments, the apparatus further comprises:

[0094] The second retrieval module is used to retrieve the second address information of the target tuple descriptor data from the local cache of the corresponding process when it receives a query request for the target tuple descriptor data of the first process or the second process. The target tuple data is tuple data that exists simultaneously in the first system directory data of the first process and the second system directory data of the second process.

[0095] The second query module is used to query target tuple descriptor data in the second shared resource pool based on the second address information.

[0096] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0097] In this embodiment, the metadata caching device and the metadata querying device in the database are presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0098] This invention also provides a computer device having the above-described features. Figure 6 The device shown is for retrieving metadata from the database.

[0099] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 7 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions that can be executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 7 Take a processor 10 as an example.

[0100] The processor 10 can be a central processor, a network processor, or a combination thereof. The processor 10 can further include hardware chips. The hardware chips can be application specific integrated circuits, programmable logic devices, or a combination thereof. The programmable logic devices can be complex programmable logic devices, field programmable logic gate arrays, general array logic, or any combination thereof.

[0101] The memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods illustrated in the above embodiments.

[0102] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function, and the like. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one disk storage device, a flash memory device, or other non-transitory solid state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, which can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0103] The memory 20 can include a volatile memory such as a random access memory, and can also include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk. The memory 20 can further include a combination of the above-mentioned types of memories.

[0104] The computer device further includes a communication interface 30 for communication of the computer device with other devices or communication networks.

[0105] The embodiments of the present application also provide a computer readable storage medium. The above-mentioned methods according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network downloading, so that the methods described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can further include a combination of the above-mentioned types of memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods illustrated in the above embodiments are implemented.

[0106] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, through the operation of the computer, can invoke or provide the method and / or technical solutions according to the present application. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source files, executable files, installation package files, etc., and accordingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0107] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method of caching metadata in a database, characterized by, The method comprises: obtaining first metadata of a first process and second metadata of a second process in a target database, the first metadata comprising first system catalog data, and the second metadata comprising second system catalog data; determining at least one target tuple data existing in both the first system catalog data and the second system catalog data; determining first target system catalog data of the first process based on the first system catalog data and the target tuple data, the first target system catalog data not containing the target tuple data; determining second target system catalog data of the second process based on the second system catalog data and the target tuple data, the second target system catalog data not containing the target tuple data; storing the target tuple data into a pre-constructed first shared resource pool, and determining first address information of the target tuple data in the first shared resource pool; storing the first address information and the first target system catalog data into a local cache of the first process, and storing the first address information and the second target system catalog data into a local cache of the second process.

2. The method of claim 1, wherein, The first metadata further comprises first relationship descriptor data, and the second metadata further comprises second relationship descriptor data, and the method further comprises: determining at least one target tuple descriptor data existing in both the first relationship descriptor data and the second relationship descriptor data; determining first target relationship descriptor data of the first process based on the first relationship descriptor data and the target tuple descriptor data, the first target relationship descriptor data not containing the target tuple descriptor data; determining second target relationship descriptor data of the second process based on the second relationship descriptor data and the target tuple descriptor data, the second target relationship descriptor data not containing the target tuple descriptor data; storing the target tuple descriptor data into a pre-constructed second shared resource pool, and determining second address information of the target tuple descriptor data in the second shared resource pool; storing the second address information and the first target relationship descriptor data into the local cache of the first process, and storing the second address information and the second target relationship descriptor data into the local cache of the second process.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: when a query request for target tuple data of the first process or the second process is received, retrieving first address information of the target tuple data from a local cache of a corresponding process, the target tuple data being tuple data existing in both first system catalog data of the first process and second system catalog data of the second process; querying the target tuple data in the first shared resource pool based on the first address information.

4. The method of claim 3, wherein, The method further comprises: when a query request for target tuple descriptor data of the first process or the second process is received, retrieving second address information of the target tuple descriptor data from a local cache of a corresponding process, the target tuple data being tuple data existing in both first system catalog data of the first process and second system catalog data of the second process; query the target tuple descriptor data in a second shared resource pool based on the second address information.

5. A caching system for metadata in a database, characterized by, The system comprises: a local cache of the first process, a local cache of the second process, a first shared resource pool, and a cache management module. The cache management module is configured to perform the cache method of metadata in the database according to any one of claims 1 to 4.

6. An apparatus for caching metadata in a database, the apparatus comprising: The apparatus comprises: an acquisition module configured to acquire first metadata of the first process and second metadata of the second process in a target database, the first metadata comprising first system catalog data, and the second metadata comprising second system catalog data; a first determination module configured to determine at least one target tuple data that exists in both the first system catalog data and the second system catalog data; a second determination module configured to determine first target system catalog data of the first process based on the first system catalog data and the target tuple data, the first target system catalog data not containing the target tuple data; a third determination module configured to determine second target system catalog data of the second process based on the second system catalog data and the target tuple data, the second target system catalog data not containing the target tuple data; a first construction module configured to store the target tuple data into a first shared resource pool constructed in advance, and determine first address information of the target tuple data in the first shared resource pool; a first storage module configured to store the first address information and the first target system catalog data into the local cache of the first process, and store the first address information and the second target system catalog data into the local cache of the second process.

7. The apparatus of claim 6, wherein, The first metadata further comprises first relation descriptor data, and the second metadata further comprises second relation descriptor data, and the apparatus further comprises: a fourth determination module configured to determine at least one target tuple descriptor data that exists in both the first relation descriptor data and the second relation descriptor data; a fifth determination module configured to determine first target relation descriptor data of the first process based on the first relation descriptor data and the target tuple descriptor data, the first target relation descriptor data not containing the target tuple descriptor data; a sixth determination module configured to determine second target relation descriptor data of the second process based on the second relation descriptor data and the target tuple descriptor data, the second target relation descriptor data not containing the target tuple descriptor data; a second construction module configured to store the target tuple descriptor data into a second shared resource pool constructed in advance, and determine second address information of the target tuple descriptor data in the second shared resource pool; a second storage module configured to store the second address information and the first target relation descriptor data into the local cache of the first process, and store the second address information and the second target relation descriptor data into the local cache of the second process.

8. A computer device, comprising: comprises: A memory and a processor, which are connected in communication with each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method for caching metadata in a database according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a computer to perform the method for caching metadata in a database according to any one of claims 1 to 4.

10. A computer program product, characterised in that, The computer readable storage medium stores computer instructions for causing a computer to perform the method for caching metadata in a database according to any one of claims 1 to 4.

Images