Caching method, system, device and equipment for metadata in database and medium

Abstract

The invention relates to the technical field of electronics, and discloses a caching method, system, device and equipment for metadata in a database and a medium, the method comprises the steps that first metadata of a first process and second metadata of a second process in a target database are obtained, the first metadata comprise first system directory data, and the second metadata comprise second system directory data; storing at least one target tuple data in the first system directory data of the first process and the second system directory data of the second process in a first shared resource pool, the first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in the corresponding local caches, and do not need to store the target tuple data in respective caches, so that the memory occupation of metadata caches in a database is effectively reduced; the problem that in the prior art, due to the fact that metadata copies are cached in each process in a redundant mode, too much memory is occupied is solved.

Description

Technical Field

[0001] The present invention relates to the field of electronic technology, and particularly to a method, system, device, equipment and medium for caching metadata in a database. Background Art

[0002] System catalog metadata is the core descriptive information stored in system tables in a database, covering the definitions and association relationships of objects such as databases, tables, columns, indexes, constraints, permissions, etc. To improve the access efficiency of system catalog metadata, the relational database system (PostgreSQL) introduces multiple cache data in each process: System Catalog Cache (CatCache / SysCache), Relation Descriptor Cache (RelCache). PostgreSQL adopts a process architecture model, and each client connection will start a backend process, and each backend process has an independent metadata cache. Since different backend processes have their own corresponding RelCache and CatCache, when the number of connections is large or the number of objects in the database is large (tables, views, indexes, etc.), the database instance will consume a large amount of memory. When the number of connections N is large and the metadata cache M is large, the PostgreSQL cluster will occupy a lot of memory (N * M), and in severe cases, it will cause Out Of Memory (OOM). Summary of the Invention

[0003] In view of this, the present invention provides a method, system, device, equipment and medium for caching metadata in a database to solve the problem of excessive memory occupation caused by redundant caching of metadata copies in each process under the multi-process architecture model of PostgreSQL in related technologies.

[0004] First aspect, the present invention provides a method for caching metadata in a database, the method comprising: obtaining first metadata of a first process and second metadata of a second process in a target database, the first metadata including first system catalog data, and the second metadata including second system catalog data; determining at least one target tuple data that exists in both the first system catalog data and the second system catalog data; determining first target system catalog data of the first process based on the first system catalog data and the target tuple data, the first target system catalog data not including the target tuple data; determining second target system catalog data of the second process based on the second system catalog data and the target tuple data, the second target system catalog data not including the target tuple data; storing the target tuple data into a pre-constructed first shared resource pool, and determining first address information of the target tuple data in the first shared resource pool; storing the first address information and the first target system catalog data into the local cache of the first process, and storing the first address information and the second target system catalog data into the local cache of the second process.

[0005] The method for caching metadata in a database provided by the present invention obtains first metadata of a first process and second metadata of a second process in a target database, the first metadata including first system catalog data, and the second metadata including second system catalog data; determines at least one target tuple data that exists in both the first system catalog data and the second system catalog data; determines first target system catalog data of the first process based on the first system catalog data and the target tuple data, the first target system catalog data not including the target tuple data; determines second target system catalog data of the second process based on the second system catalog data and the target tuple data, the second target system catalog data not including the target tuple data; stores the target tuple data into a pre-constructed first shared resource pool, and determines first address information of the target tuple data in the first shared resource pool; stores the first address information and the first target system catalog data into the local cache of the first process, and stores the first address information and the second target system catalog data into the local cache of the second process. The method provided by the present invention stores at least one target tuple data that exists in both the first system catalog data of the first process and the second system catalog data of the second process into the first shared resource pool. The first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in their respective local caches, and do not need to separately store the target tuple data in their respective local caches, effectively reducing the memory occupation of metadata caching in the database, and solving the problem of excessive memory occupation caused by redundant caching of metadata copies by each process in the prior art.

[0006] In an alternative embodiment, the first metadata further includes first relationship descriptor data, the second metadata further includes second relationship descriptor data, and the method further includes: determining at least one target tuple descriptor data that exists in both the first relationship descriptor data and the second relationship descriptor data; determining first target relationship descriptor data of a first process based on the first relationship descriptor data and the target tuple descriptor data, the first target relationship descriptor data not including the target tuple descriptor data; determining second target relationship descriptor data of a second process based on the second relationship descriptor data and the target tuple descriptor data, the second target relationship descriptor data not including the target tuple descriptor data; storing the target tuple descriptor data into a pre-constructed second shared resource pool, and determining second address information of the target tuple descriptor data in the second shared resource pool; storing the second address information and the first target relationship descriptor data into a local cache of the first process, and storing the second address information and the second target relationship descriptor data into a local cache of the second process.

[0007] The method provided in this alternative embodiment stores at least one target tuple descriptor data that exists in both the first relationship descriptor data of the first process and the second relationship descriptor data of the second process into the second shared resource pool. The first process and the second process only need to store the address information of the target tuple descriptor data in the second shared resource pool in their corresponding local caches, and do not need to separately store the target tuple descriptor data in their respective local caches, effectively alleviating the memory pressure and solving the problem of excessive memory occupation caused by each process redundantly caching metadata copies in the prior art.

[0008] In an alternative embodiment, the method further includes: when a query request for target tuple data of the first process or the second process is received, retrieving first address information of the target tuple data from the local cache of the corresponding process, the target tuple data being tuple data that exists in both the first system catalog data of the first process and the second system catalog data of the second process; querying the target tuple data in the first shared resource pool based on the first address information.

[0009] In an alternative embodiment, the method further includes: when a query request for target tuple descriptor data of the first process or the second process is received, retrieving second address information of the target tuple descriptor data from the local cache of the corresponding process, the target tuple data being tuple data that exists in both the first system catalog data of the first process and the second system catalog data of the second process; querying the target tuple descriptor data in the second shared resource pool based on the second address information.

[0010] In a second aspect, the present invention provides a cache system for metadata in a database, including: a local cache of a first process, a local cache of a second process, a first shared resource pool, and a cache management module; the cache management module is configured to execute the cache method for metadata in the database according to the first aspect or any corresponding embodiment thereof.

[0011] The cache system for metadata in the database provided by the present invention includes a local cache of a first process, a local cache of a second process, a first shared resource pool, and a cache management module; the cache management module stores at least one target tuple data that exists in both the first system directory data of the first process and the second system directory data of the second process into the first shared resource pool. The first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in their corresponding local caches, and do not need to store the target tuple data separately in their respective local caches, effectively reducing the memory occupancy of metadata caching in the database and solving the problem of excessive memory occupancy caused by redundant caching of metadata copies by each process in the prior art.

[0012] In a third aspect, the present invention provides a cache device for metadata in a database. The device includes: an acquisition module, configured to acquire first metadata of a first process and second metadata of a second process in a target database, where the first metadata includes first system directory data and the second metadata includes second system directory data; a first determination module, configured to determine at least one target tuple data that exists in both the first system directory data and the second system directory data; a second determination module, configured to determine first target system directory data of the first process based on the first system directory data and the target tuple data, where the first target system directory data does not include the target tuple data; a third determination module, configured to determine second target system directory data of the second process based on the second system directory data and the target tuple data, where the second target system directory data does not include the target tuple data; a first construction module, configured to store the target tuple data into a pre-constructed first shared resource pool and determine first address information of the target tuple data in the first shared resource pool; a first storage module, configured to store the first address information and the first target system directory data into the local cache of the first process, and store the first address information and the second target system directory data into the local cache of the second process.

[0013] In some alternative embodiments, the above-mentioned device further includes: the first metadata further includes first relationship descriptor data, the second metadata further includes second relationship descriptor data, and the device further includes: a fourth determination module, configured to determine at least one target tuple descriptor data that exists in both the first relationship descriptor data and the second relationship descriptor data; a fifth determination module, configured to determine first target relationship descriptor data of a first process based on the first relationship descriptor data and the target tuple descriptor data, where the first target relationship descriptor data does not include the target tuple descriptor data; a sixth determination module, configured to determine second target relationship descriptor data of a second process based on the second relationship descriptor data and the target tuple descriptor data, where the second target relationship descriptor data does not include the target tuple descriptor data; a second construction module, configured to store the target tuple descriptor data into a pre-constructed second shared resource pool, and determine second address information of the target tuple descriptor data in the second shared resource pool; a second storage module, configured to store the second address information and the first target relationship descriptor data into the local cache of the first process, and store the second address information and the second target relationship descriptor data into the local cache of the second process.

[0014] In a fourth aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the method for caching metadata in a database according to the first aspect or any corresponding embodiment thereof, or execute the method for querying metadata in a database according to the second aspect or its corresponding embodiment.

[0015] In a fifth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the method for caching metadata in a database according to the first aspect or any corresponding embodiment thereof.

[0016] In a sixth aspect, the present invention provides a computer program product, including computer instructions, and the computer instructions are used to cause a computer to execute the method for caching metadata in a database according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1It is a flowchart of a method for caching metadata in a database according to an embodiment of the present invention;

[0019] Figure 2 It is a flowchart of a method for caching metadata in another database according to an embodiment of the present invention;

[0020] Figure 3 It is a schematic diagram of a global system cache architecture according to an embodiment of the present invention;

[0021] Figure 4 It is a schematic diagram of a global relationship cache architecture according to an embodiment of the present invention;

[0022] Figure 5 It is a block diagram of the structure of a metadata caching system in a database according to an embodiment of the present invention

[0023] Figure 6 It is a block diagram of the structure of a metadata caching device in a database according to an embodiment of the present invention;

[0024] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Specific embodiments

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0026] In the related art, PostgreSQL adopts a process architecture model, and each client connection will start a backend process. Each backend process has an independent metadata cache. Since different backend processes have their respective RelCache and CatCache, when the number of connections is large or the number of objects in the database is large (such as tables, views, indexes, etc.), the database instance will consume a large amount of memory. When the number of connections N is large and the metadata cache M is large, the PostgreSQL cluster will occupy a lot of memory (N * M), and in severe cases, it will cause out-of-memory (OOM).

[0027] In view of this, a method for caching metadata in a database provided by an embodiment of the present application can be applied to a server to implement caching of metadata in the database. In the method provided by the embodiment of the present application, at least one target tuple data that exists in both the first system catalog data of the first process and the second system catalog data of the second process is stored in the first shared resource pool. The first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in their respective local caches, and do not need to separately store the target tuple data in their respective local caches, effectively reducing the memory occupancy of metadata caching in the database and solving the problem of excessive memory occupancy caused by redundant caching of metadata copies by each process in the prior art.

[0028] According to an embodiment of the present invention, an embodiment of a method for caching metadata in a database is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0029] In this embodiment, a method for caching metadata in a database is provided, which can be used for the above-mentioned server. Figure 1 It is a flowchart of a method for caching metadata in a database according to an embodiment of the present invention, as Figure 1 shown, and the process includes the following steps:

[0030] Step S101, obtain the first metadata of the first process and the second metadata of the second process in the target database, where the first metadata includes first system catalog data and the second metadata includes second system catalog data.

[0031] Exemplarily, the first process and the second process can be application program processes running on the server side (backend), responsible for processing core functions such as business logic, data interaction, and resource scheduling. In the embodiment of the present application, the first process and the second process are different backend processes. System Catalog (SysCache) data refers to the core component in the Database Management System (DBMS) that stores metadata, and is used to record database structure, object definitions, and their associated information.

[0032] Step S102, determine at least one target tuple data that exists in both the first system catalog data and the second system catalog data.

[0033] Exemplarily, both the first system catalog data and the second system catalog data contain multiple tuple data. In a relational database (such as PostgreSQL, MySQL), a tuple is the basic data unit of the relational model, corresponding to "one row of data" in a table). In the embodiments of the present application, due to the complex data structure relationship of SysCache, various pointers point to each other (for example, CatCTup has pointers to CatCache and CatCList. CatCTup is one of the core data structures of the catalog cache, and its design purpose is to efficiently manage the cache entries of metadata. CatCList is used to store the result list of partial match queries), and the content in these data structures is updated frequently. If SysCache is shared as a whole, it is necessary to lock to prevent concurrent operations. Leaving aside the implementation difficulty, since SysCache is a high-frequency access module, the performance degradation caused by lock contention is unacceptable. Therefore, the scheme of sharing SysCache as a whole is not adopted. After analysis, the data pointed to by the tuple field is the part that occupies the largest memory in SysCache, and this data does not change under a specific key and specific version. Therefore, the tuple data is placed in the shared memory, and the local process pointer points to the tuple data, and the same tuple data of different processes is shared. Therefore, the embodiments of the present application need to determine the target tuple data that exists in both the first system catalog data and the second system catalog data.

[0034] Step S103, determine the first target system catalog data of the first process based on the first system catalog data and the target tuple data, and the first target system catalog data does not include the target tuple data.

[0035] Exemplarily, in the embodiments of the present application, the other data in the first system catalog data except the target tuple data is used as the first target system catalog data.

[0036] Step S104, determine the second target system catalog data of the second process based on the second system catalog data and the target tuple data, and the second target system catalog data does not include the target tuple data.

[0037] Exemplarily, in the embodiments of the present application, the other data in the second system catalog data except the target tuple data is used as the second target system catalog data.

[0038] Step S105, store the target tuple data into a pre-constructed first shared resource pool, and determine the first address information of the target tuple data in the first shared resource pool.

[0039] Exemplarily, the first shared resource pool is a pre-constructed storage area for storing tuple data shared by different processes. In the embodiments of the present application, the first address information may include, but is not limited to, a tuple pointer. A Tuple Pointer is an identifier used to locate the physical or logical address of a certain tuple (i.e., a row record in the table) in the storage layer in the table. It is the core mechanism for a database to achieve efficient data access and is commonly found in the storage engines and index designs of relational databases.

[0040] Step S106, store the first address information and the first target system directory data in the local cache of the first process, and store the first address information and the second target system directory data in the local cache of the second process.

[0041] The method for caching metadata in the database provided in this embodiment stores at least one target tuple data that exists in both the first system directory data of the first process and the second system directory data of the second process in the first shared resource pool. The first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in their corresponding local caches, and do not need to separately store the target tuple data in their respective local caches, effectively reducing the memory occupancy of metadata caching in the database and solving the problem of excessive memory occupancy caused by each process redundantly caching metadata copies in the prior art.

[0042] In this embodiment, a method for caching metadata in a database is provided, which can be used for the above-mentioned server. Figure 2 It is a flowchart of the method for caching metadata in a database according to an embodiment of the present invention, as Figure 2 shown. The process includes the following steps:

[0043] Step S201, obtain the first metadata of the first process and the second metadata of the second process in the target database. The first metadata includes the first system directory data, and the second metadata includes the second system directory data. For details, please refer to Figure 1 Step S101 of the embodiment shown, which will not be elaborated here.

[0044] Step S202, determine at least one target tuple data that exists in both the first system directory data and the second system directory data. For details, please refer to Figure 1 Step S102 of the embodiment shown, which will not be elaborated here.

[0045] Step S203, determine the first target system directory data of the first process based on the first system directory data and the target tuple data. The first target system directory data does not include the target tuple data. For details, please refer to Figure 1 Step S103 of the embodiment shown, which will not be elaborated here.

[0046] Step S204: Determine the second target system directory data of the second process based on the second system directory data and the target tuple data. The second target system directory data does not include the target tuple data. For details, please refer to Figure 1 Step S104 of the embodiment shown, which will not be elaborated here.

[0047] Step S205: Store the target tuple data into a pre-constructed first shared resource pool, and determine the first address information of the target tuple data in the first shared resource pool. For details, please refer to Figure 1 Step S105 of the embodiment shown, which will not be elaborated here.

[0048] Step S206: Store the first address information and the first target system directory data into the local cache of the first process, and store the first address information and the second target system directory data into the local cache of the second process. For details, please refer to Figure 1 Step S106 of the embodiment shown, which will not be elaborated here.

[0049] Step S207: Determine at least one target tuple descriptor data that exists in both the first relation descriptor data and the second relation descriptor data.

[0050] Exemplarily, the relation descriptor data (Relation Descriptor Data) contains multiple element descriptor (Tuple Descriptor, TupleDesc) data. TupleDesc is a structure mainly used to describe the information of each attribute (column) in a tuple, including the name, data type, length, etc. of the attribute. In the embodiments of the present application, like SysCache, the entire hash table (HashTable) of RelCache is not selected for sharing. After analysis, the data pointed to by the TupleDesc field is the part that occupies the largest memory in RelCache, and this data does not change under a specific version. Therefore, the TupleDesc data is placed in the shared memory, and the local process pointer points to the TupleDesc data, and the same TupleDesc data of different processes is shared. Therefore, in the embodiments of the present application, it is necessary to determine the target tuple descriptor data that exists in the relation descriptor data of different processes.

[0051] Step S208: Determine the first target relation descriptor data of the first process based on the first relation descriptor data and the target tuple descriptor data. The first target relation descriptor data does not include the target tuple descriptor data.

[0052] Exemplarily, in the embodiments of the present application, the data in the first relationship descriptor data except the target tuple descriptor data is used as the first target relationship descriptor data.

[0053] Step S209, determine the second target relationship descriptor data of the second process based on the second relationship descriptor data and the target tuple descriptor data, and the second target relationship descriptor data does not include the target tuple descriptor data.

[0054] Exemplarily, in the embodiments of the present application, the data in the second relationship descriptor data except the target tuple descriptor data is used as the second target relationship descriptor data.

[0055] Step S2010, store the target tuple descriptor data into a pre-constructed second shared resource pool, and determine the second address information of the target tuple descriptor data in the second shared resource pool.

[0056] Exemplarily, the second shared resource pool is a pre-constructed storage area for storing the target tuple descriptor data. In the embodiments of the present application, the second address information may include, but is not limited to, a TupleDesc pointer. In a database system, the TupleDesc pointer plays a key role. It points to the TupleDesc structure, and this structure is used to describe the structure information of a tuple (a row in a table).

[0057] Step S2011, store the second address information and the first target relationship descriptor data into the local cache of the first process, and store the second address information and the second target relationship descriptor data into the local cache of the second process.

[0058] In some alternative embodiments, the above method further includes:

[0059] Step a1, when a query request for the target tuple data of the first process or the second process is received, retrieve the first address information of the target tuple data from the local cache of the corresponding process, where the target tuple data is the tuple data that exists in both the first system directory data of the first process and the second system directory data of the second process.

[0060] Exemplarily, in the embodiments of the present application, the query request for the target tuple data may include the unique identification information of the target tuple data to be queried, and based on the unique identification information, retrieve the tuple pointer of the target tuple data.

[0061] Step a2, query the target tuple data in the first shared resource pool based on the first address information.

[0062] Exemplarily, in the embodiments of the present application, query the target tuple data in the first shared resource pool based on the tuple pointer of the target tuple data.

[0063] Step a3, when a query request for the target tuple descriptor data of the first process or the second process is received, retrieve the second address information of the target tuple descriptor data from the local cache of the corresponding process, where the target tuple data is the tuple data that exists in both the first system directory data of the first process and the second system directory data of the second process.

[0064] Exemplarily, in the embodiment of the present application, the query request for the target tuple descriptor data may include the unique identification information of the target tuple descriptor data to be queried, and based on the unique identification information, retrieve the TupleDesc pointer of the target tuple descriptor data.

[0065] Step a4, query the target tuple descriptor data in the second shared resource pool based on the second address information.

[0066] Exemplarily, in the embodiment of the present application, query the target tuple descriptor data in the second shared resource pool based on the TupleDesc pointer of the target tuple descriptor data.

[0067] The following specifically describes the method for caching metadata in the database provided by the present application through a specific embodiment.

[0068] Embodiment:

[0069] (1) Global System Cache (Global SysCache) architecture design: The data structure relationship of the entire SysCache system is complex, and various pointers point to each other (for example, CatCTup has pointers to CatCache and catclist) (CatCList is used to store the result list of partial match queries), and the content in these data structures is updated frequently. If SysCache is shared as a whole, it is necessary to lock to prevent concurrent operations. Leaving aside the implementation difficulty, since SysCache is a high-frequency access module, the performance degradation caused by lock contention is unacceptable. Therefore, the scheme of sharing SysCache as a whole is not adopted. After analysis, the data pointed to by the tuple field is the part with the largest memory occupancy in SysCache, and this data does not change under a specific key and specific version. Therefore, the tuple data is placed in the shared memory, and the local process pointer points to the tuple data, and the same tuple data of different processes is shared. For two processes A and B, the schematic diagram of the global system cache architecture is as Figure 3 shown. Build a HashTable in the shared memory, and place the same tuple data of different processes in the shared memory, and the local process points to it through a pointer. The key of the HashTable is to calculate the hashvalue of the memory data in this segment.

[0070] (2) Global Relationship Cache (GlobalRelCache) Architecture Design: Similar to SysCache, it does not choose to share the entire HashTable of RelCache.

[0071] After analysis, the data pointed to by the TupleDesc field is the part that occupies the largest memory in RelCache, and this data does not change under specific versions. Therefore, the TupleDesc data is placed in the shared memory, and the local process pointer points to the TupleDesc data, and the same TupleDesc data of different processes is shared. The schematic diagram of the global relationship cache architecture is as Figure 4 shown.

[0072] A HashTable is constructed in the shared memory. The same TupleDesc data of different processes is placed in the shared memory, and the local process points to the TupleDesc data through a pointer. The key of the HashTable is the hashvalue calculated for the memory data of this segment.

[0073] The embodiment of the present application also provides a cache system for metadata in a database, as Figure 5 shown. The system includes: the local cache 501 of the first process, the local cache 502 of the second process, the first shared resource pool 503, and the cache management module 504; the cache management module 504 is used to execute the cache method for metadata in the database in the above embodiment. Exemplarily, for specific details, refer to the description of the relevant content in the above embodiment, which will not be elaborated here.

[0074] The cache system for metadata in the database provided by the embodiment of the present application includes the local cache of the first process, the local cache of the second process, the first shared resource pool, and the cache management module; the cache management module stores at least one target tuple data that exists in both the first system catalog data of the first process and the second system catalog data of the second process into the first shared resource pool. The first process and the second process only need to store the address information of the target tuple data in the first shared resource pool in the corresponding local caches, and do not need to store the target tuple data separately in their respective local caches, effectively reducing the memory occupancy of metadata caching in the database and solving the problem of excessive memory occupancy caused by redundant caching of metadata copies by each process in the prior art.

[0075] In some alternative embodiments, the above system further includes a shared memory base. The hash table of the shared memory base manages the shared data of the shared resource pool and provides interfaces for looking up, inserting, and deleting the shared resource pool. The shared resources include a reference count: when the backend process inserts a shared resource, the reference count is initialized to 1; when the backend process looks up and then references the shared resource, the reference count is incremented by 1; when the backend process releases the shared resource, the reference count is decremented by 1. If the reference count is decremented to 0, the shared resource is deleted. The shared resource pool includes 64 hash tables, and each hash table has a read-write lock: to modify (insert or delete) the hash table, a write lock is required, and to look up the hash table, a read lock is required.

[0076] In this embodiment, a cache device for metadata in a database is further provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated here. As used hereinafter, the term "module" may be a combination of software and / or hardware that can implement a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0077] This embodiment provides a cache device for metadata in a database, as Figure 6 shown, including:

[0078] An acquisition module 601, configured to acquire first metadata of a first process and second metadata of a second process in a target database, where the first metadata includes first system catalog data, and the second metadata includes second system catalog data;

[0079] A first determination module 602, configured to determine at least one target tuple data that exists in both the first system catalog data and the second system catalog data;

[0080] A second determination module 603, configured to determine first target system catalog data of the first process based on the first system catalog data and the target tuple data, where the first target system catalog data does not include the target tuple data;

[0081] A third determination module 604, configured to determine second target system catalog data of the second process based on the second system catalog data and the target tuple data, where the second target system catalog data does not include the target tuple data;

[0082] A first construction module 605, configured to store the target tuple data into a pre-constructed first shared resource pool and determine first address information of the target tuple data in the first shared resource pool;

[0083] The first storage module 606 is configured to store the first address information and the first target system directory data into the local cache of the first process, and store the first address information and the second target system directory data into the local cache of the second process.

[0084] In some alternative embodiments, the first metadata further includes first relationship descriptor data, the second metadata further includes second relationship descriptor data, and the apparatus further includes:

[0085] A fourth determination module, configured to determine at least one target tuple descriptor data that exists in both the first relationship descriptor data and the second relationship descriptor data;

[0086] A fifth determination module, configured to determine first target relationship descriptor data of the first process based on the first relationship descriptor data and the target tuple descriptor data, where the first target relationship descriptor data does not include the target tuple descriptor data;

[0087] A sixth determination module, configured to determine second target relationship descriptor data of the second process based on the second relationship descriptor data and the target tuple descriptor data, where the second target relationship descriptor data does not include the target tuple descriptor data;

[0088] A second construction module, configured to store the target tuple descriptor data into a pre-constructed second shared resource pool, and determine second address information of the target tuple descriptor data in the second shared resource pool;

[0089] A second storage module, configured to store the second address information and the first target relationship descriptor data into the local cache of the first process, and store the second address information and the second target relationship descriptor data into the local cache of the second process.

[0090] In some alternative embodiments, the above apparatus further includes:

[0091] A first retrieval module, configured to, when receiving a query request for target tuple data of the first process or the second process, retrieve first address information of the target tuple data from the local cache of the corresponding process, where the target tuple data is tuple data that exists in both the first system directory data of the first process and the second system directory data of the second process;

[0092] A first query module, configured to query the target tuple data in the first shared resource pool based on the first address information.

[0093] In some alternative embodiments, the above apparatus further includes:

[0094] A second retrieval module, configured to retrieve second address information of target tuple descriptor data from a local cache of a corresponding process when receiving a query request for the target tuple descriptor data of a first process or a second process, where the target tuple data is tuple data that exists in both first system directory data of the first process and second system directory data of the second process;

[0095] A second query module, configured to query the target tuple descriptor data in a second shared resource pool based on the second address information.

[0096] The further function descriptions of the above-mentioned various modules and units are the same as those in the corresponding embodiments above, and will not be elaborated here.

[0097] The cache device for metadata in the database and the query device for metadata in the database in this embodiment are presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0098] An embodiment of the present invention further provides a computer device having the above-mentioned Figure 6 retrieval device for metadata in the database as shown.

[0099] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of a computer device provided by an optional embodiment of the present invention. As Figure 7 shown, the computer device includes: one or more processors 10, a memory 20, and an interface for connecting each component, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executable within the computer device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 7 In

[0100] The processor 10 may be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 may further include a hardware chip. The above-mentioned hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device may be a complex programmable logic device, a field programmable gate array, a generic array logic, or any combination thereof.

[0101] Among them, the memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiments.

[0102] The memory 20 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 may optionally include a memory remotely provided with respect to the processor 10, and these remote memories may be connected to the computer device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0103] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive; the memory 20 may further include a combination of the above types of memories.

[0104] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

[0105] The embodiments of the present invention also provide a computer-readable storage medium. The method according to the embodiments of the present invention may be implemented in hardware, firmware, or may be implemented as computer code recorded on a storage medium, or may be implemented as computer code originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored as such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium may further include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0106] A part of the present invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can call or provide the methods and / or technical solutions according to the present invention through the operations of the computer. Those skilled in the art should understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0107] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for caching metadata in a database, characterized in that The method includes: Obtain the first metadata of the first process and the second metadata of the second process in the target database, where the first metadata includes first system catalog data and the second metadata includes second system catalog data; Determine at least one target tuple data that exists in both the first system catalog data and the second system catalog data; Based on the first system catalog data and the target tuple data, determine the first target system catalog data of the first process, where the first target system catalog data does not include the target tuple data; Based on the second system catalog data and the target tuple data, determine the second target system catalog data of the second process, where the second target system catalog data does not include the target tuple data; Store the target tuple data into a pre-constructed first shared resource pool, and determine the first address information of the target tuple data in the first shared resource pool; Store the first address information and the first target system catalog data into the local cache of the first process, and store the first address information and the second target system catalog data into the local cache of the second process.

2. The method according to claim 1, wherein The first metadata further includes first relationship descriptor data, and the second metadata further includes second relationship descriptor data. The method further includes: Determine at least one target tuple descriptor data that exists in both the first relationship descriptor data and the second relationship descriptor data; Based on the first relationship descriptor data and the target tuple descriptor data, determine the first target relationship descriptor data of the first process, where the first target relationship descriptor data does not include the target tuple descriptor data; Based on the second relationship descriptor data and the target tuple descriptor data, determine the second target relationship descriptor data of the second process, where the second target relationship descriptor data does not include the target tuple descriptor data; Store the target tuple descriptor data into a pre-constructed second shared resource pool, and determine the second address information of the target tuple descriptor data in the second shared resource pool; Store the second address information and the first target relationship descriptor data into the local cache of the first process, and store the second address information and the second target relationship descriptor data into the local cache of the second process.

3. The method according to claim 1 or 2, characterized in that, The method further includes: When a query request for the target tuple data of the first process or the second process is received, retrieve the first address information of the target tuple data from the local cache of the corresponding process, where the target tuple data is the tuple data that exists in both the first system catalog data of the first process and the second system catalog data of the second process; Query the target tuple data in the first shared resource pool based on the first address information.

4. The method according to claim 3, wherein The method further includes: When a query request for the target tuple descriptor data of the first process or the second process is received, retrieve the second address information of the target tuple descriptor data from the local cache of the corresponding process, where the target tuple data is the tuple data that exists in both the first system catalog data of the first process and the second system catalog data of the second process; Query the target tuple descriptor data in the second shared resource pool based on the second address information.

5. A cache system for metadata in a database, characterized in that, The system includes: the local cache of the first process, the local cache of the second process, the first shared resource pool, and the cache management module; The cache management module is used to execute the cache method for metadata in the database according to any one of claims 1 to 4.

6. A cache device for metadata in a database, characterized in that, The device includes: An acquisition module, configured to acquire first metadata of a first process and second metadata of a second process in a target database, where the first metadata includes first system catalog data, and the second metadata includes second system catalog data; A first determination module, configured to determine at least one target tuple data that exists in both the first system catalog data and the second system catalog data; A second determination module, configured to determine first target system catalog data of the first process based on the first system catalog data and the target tuple data, where the first target system catalog data does not include the target tuple data; A third determination module, configured to determine second target system catalog data of the second process based on the second system catalog data and the target tuple data, where the second target system catalog data does not include the target tuple data; A first construction module, configured to store the target tuple data into a pre-constructed first shared resource pool, and determine first address information of the target tuple data in the first shared resource pool; A first storage module, configured to store the first address information and the first target system catalog data into the local cache of the first process, and store the first address information and the second target system catalog data into the local cache of the second process.

7. The device according to claim 6, characterized in that, The first metadata further includes first relationship descriptor data, the second metadata further includes second relationship descriptor data, and the device further includes: A fourth determination module, configured to determine at least one target tuple descriptor data that exists in both the first relationship descriptor data and the second relationship descriptor data; A fifth determination module, configured to determine first target relationship descriptor data of the first process based on the first relationship descriptor data and the target tuple descriptor data, where the first target relationship descriptor data does not include the target tuple descriptor data; A sixth determination module, configured to determine second target relationship descriptor data of the second process based on the second relationship descriptor data and the target tuple descriptor data, where the second target relationship descriptor data does not include the target tuple descriptor data; A second construction module, configured to store the target tuple descriptor data into a pre-constructed second shared resource pool, and determine second address information of the target tuple descriptor data in the second shared resource pool; A second storage module, configured to store the second address information and the first target relationship descriptor data into the local cache of the first process, and store the second address information and the second target relationship descriptor data into the local cache of the second process.

8. A computer device, characterized in that, Includes: A memory and a processor, which are communicatively connected to each other. Computer instructions are stored in the memory, and the processor executes the computer instructions to execute the method for caching metadata in the database according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the method for caching metadata in the database according to any one of claims 1 to 4.

10. A computer program product, characterized in that, It includes computer instructions, and the computer instructions are used to cause a computer to execute the method for caching metadata in the database according to any one of claims 1 to 4.

Images