Data caching method and device, electronic equipment and storage medium

Abstract

The invention provides a data caching method and device, electronic equipment and a storage medium, and the method comprises the steps: if the space occupied by persistently cached data on local storage equipment is greater than a persistent cache threshold value, deleting part of data in the persistently cached data on the local storage equipment, sending the deleted data to a local storage device so that the space occupied by the persistent cached data on the local storage device is not greater than a persistent cache threshold value, and transferring metadata of the deleted data from a cache region in the memory to a standby cache region; if the space occupied by the persistent cached data on the local storage device is smaller than the persistent cache threshold value, caching the data to which part or all of the metadata in the standby cache region in the memory belong to the local storage device, and transferring the metadata of the cached data to the cache region from the standby cache region in the memory; wherein the persistent cache threshold value is a space threshold value which is distributed by the database node on the local storage device and is used for persistent cache.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of database technology, and in particular, to a data caching method and device, an electronic device, and a storage medium. Background Art

[0002] With the exponential growth of data in the computer field and the development of database technology, a shared storage method has gradually emerged. That is, by setting up a shared data layer and multiple database nodes that communicate with it, user data and system internal metadata, etc. can be stored in the shared data layer for sharing by multiple database nodes. The database nodes will cache part of the data in the shared data layer locally and provide data services to the outside world based on it.

[0003] In addition to data caching, the local storage space of database nodes also needs to perform many other functions, and the data volume requirements of the business on the database nodes will also change, so the space allocated by the database nodes for persistent cache changes dynamically. However, in related technologies, database nodes do not implement persistent cache data well in scenarios where the space changes dynamically, and the data hit rate of persistent cache still needs to be improved. Summary of the invention

[0004] In view of this, one or more embodiments of the present specification provide a data caching method and device, an electronic device, and a storage medium.

[0005] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of one or more embodiments of this specification, a data caching method is proposed, the method comprising: If the space occupied by the data in the persistent cache on the local storage device is greater than the persistent cache threshold, some of the data in the persistent cache on the local storage device is deleted so that the space occupied by the data in the persistent cache on the local storage device is not greater than the persistent cache threshold, and the metadata of the deleted data is transferred from the in-memory cache area to the backup cache area; If the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold, cache the data belonging to part or all of the metadata in the backup cache area in the memory to the local storage device, and transfer the metadata of the cached data from the backup cache area in the memory to the cache area; The persistent cache threshold is a space threshold allocated by the database node on the local storage device for persistent cache.

[0006] In a possible embodiment of this specification, the method further includes: If the space required for the data to be cached currently is greater than the remaining space in the space used for persistent caching on the local storage device, part of the data in the persistent cache on the local storage device is deleted to cache the data to be cached currently to the local storage device, and the metadata of the deleted data is transferred from the cache area in the memory to the backup cache area.

[0007] In a possible embodiment of the present specification, if the space occupied by the persistent cache data on the local storage device is less than the persistent cache threshold, the data belonging to part or all of the metadata in the backup cache area in the memory is cached on the local storage device, and the metadata of the cached data is transferred from the backup cache area in the memory to the cache area, including: If the space occupied by the persistent cached data on the local storage device is less than the persistent cache threshold, the data belonging to some or all of the metadata in the backup cache area in the memory will be cached by the shared data layer to the local storage device, and the metadata of the cached data will be transferred from the backup cache area in the memory to the cache area.

[0008] In a possible embodiment of the present specification, if the space occupied by the persistent cache data on the local storage device is less than the persistent cache threshold, caching the data belonging to part or all of the metadata in the backup cache area in the memory on the local storage device includes: If the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold, there is metadata in the backup cache area in the memory, and the space required for the data belonging to the metadata in the backup cache area is less than the remaining space in the space used for persistent cache on the local storage device, then cache the data belonging to all the metadata in the backup cache area to the local storage device; If the space occupied by the data of the persistent cache on the local storage device is less than the persistent cache threshold, there is metadata in the backup cache area in the memory, and the space required for the data belonging to the metadata in the backup cache area is greater than the remaining space in the space used for persistent cache on the local storage device, then the data belonging to part of the metadata in the backup cache area will be cached to the local storage device.

[0009] In a possible embodiment of the present specification, the backup cache area includes a latest access list and a frequently accessed list; and caching data belonging to a portion of metadata in the backup cache area to the local storage device includes: Based on the principle of giving priority to transferring the data belonging to the metadata in the frequently accessed list, the data belonging to part of the metadata in the backup cache area are cached to the local storage device.

[0010] In a possible embodiment of the present specification, the metadata includes access popularity of the data to which it belongs; and caching the data to which the metadata in the backup cache area belongs to the local storage device includes: Based on the principle of giving priority to transferring data with high access popularity, data belonging to part of the metadata in the backup cache area is cached to the local storage device.

[0011] In a possible embodiment of the present specification, the step of transferring metadata of the deleted data in the memory from the cache area to the standby cache area includes: If the remaining space in the backup cache area in the memory is less than the space required for the metadata of the deleted data, the space allocated to the backup cache area in the memory is adjusted to be no less than the space required for the metadata of the deleted data, and the metadata of the deleted data is transferred from the cache area to the backup cache area.

[0012] In a possible embodiment of the present specification, if the space occupied by the persistent cache data on the local storage device is less than the persistent cache threshold, the data belonging to part or all of the metadata in the backup cache area in the memory is cached on the local storage device, and the metadata of the cached data is transferred from the backup cache area in the memory to the cache area, including: If an extension node is added to the database node so that the space occupied by the persistently cached data on the local storage device of the database node is less than the persistent cache threshold, the metadata in the backup cache area in the memory of the database node is sent to the extension node, so that the extension node caches the data belonging to part or all of the received metadata on the local storage device, adds the metadata of the cached data to the cache area in the memory, and adds the remaining metadata to the backup cache area in the memory.

[0013] According to a second aspect of one or more embodiments of this specification, a data cache device is provided, which is applied to a database node. The device includes: A shrinking module is used to delete part of the data in the persistent cache on the local storage device if the space occupied by the data in the persistent cache on the local storage device is greater than the persistent cache threshold, so that the space occupied by the data in the persistent cache on the local storage device is not greater than the persistent cache threshold, and transfer the metadata of the deleted data from the in-memory cache area to the backup cache area; The expansion module is used to cache the data belonging to part or all of the metadata in the backup cache area in the memory on the local storage device if the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold, and transfer the metadata of the cached data from the backup cache area in the memory to the cache area; The persistent cache threshold is a space threshold allocated by the database node on the local storage device for persistent cache.

[0014] According to a third aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the method described in the first aspect when executed by a processor.

[0015] According to a fourth aspect of one or more embodiments of this specification, an electronic device is provided, including: processor; a memory for storing processor-executable instructions; The processor implements the method as described in the first aspect by running the executable instructions.

[0016] According to a fifth aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0017] The technical solutions provided by the embodiments of this specification may have the following beneficial effects: In the data caching method provided in the embodiment of this specification, the database node configures a cache area and a backup cache area in the memory. When the database node persists the cached data, it stores the data on a local storage device such as a disk, and loads the metadata of the cached data into the cache area. When the space used for persistent caching in the local storage device is reduced, part of the data in the persistent cache can be deleted, and the metadata of the deleted data can be transferred from the cache area to the backup cache area. When the space used for persistent caching in the local storage device of the database node is expanded, the data to which the metadata in the backup cache area belongs can be persistently cached to the local storage device, and the metadata of the cached data can be transferred from the backup cache area to the cache area. It can be seen that the data to which the metadata in the backup cache area belongs is the data that the database node wants to cache but cannot cache due to limited space. Recording the metadata of this part of the data can increase the persistent cached data in a targeted manner when the space increases, so that the added data has a higher hit rate, thereby avoiding the problem of low hit rate caused by randomly increasing the persistent cached data when the space increases in the related art. The higher hit rate makes the data service of the database node have lower latency, higher efficiency and better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a structural diagram of a data system provided by an exemplary embodiment.

[0019] Figure 2is a flow chart of a data caching method provided by an exemplary embodiment.

[0020] Figure 3 is a flowchart of a data caching method provided by another exemplary embodiment.

[0021] Figure 4 It is a structural schematic diagram of a device provided by an exemplary embodiment.

[0022] Figure 5 is a block diagram of a data cache device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0025] First, some concepts involved in this specification are explained.

[0026] Persistent caching: refers to storing cached data in a persistent storage medium so that the data can be reloaded and used after the program is closed or the power is off. Specifically, under the shared storage architecture, the database node needs to load some hot data into the local memory for caching to avoid the delay caused by the database node having to load data from the shared storage in real time every time it accesses the data. However, the amount of hot data that needs to be cached is large, and the memory space is very limited and has many uses. Therefore, the hot data of the shared data layer is persistently stored on the persistent storage medium such as the local disk of the database node to achieve persistent caching.

[0027] Spatial elasticity: refers to the ability of database nodes to automatically adjust the size of the space used for persistent cache in the persistent storage medium as needed to improve the flexibility of using persistent media space.

[0028] Object storage: Object storage is a data management architecture that treats data as objects rather than files or blocks to efficiently and flexibly store and retrieve massive amounts of unstructured data. Its "everything is an object" concept emphasizes that all data, whether it is documents, pictures, videos or logs, can be processed as objects, which helps improve access efficiency and scalability.

[0029] In the related art, when the database node under the shared storage architecture is reduced in capacity, that is, when the disk space allocated for persistent cache is reduced, the redundant data in the persistent cached data is deleted, and its metadata is deleted synchronously; when the database node under the shared storage architecture is expanded, that is, when the disk space allocated for persistence is expanded, part of the data in the shared data layer is randomly added to the local disk. This makes the hit rate of the database node low when accessing data, that is, the hot data that the database node needs to access is often not persistently cached to the local disk in advance. In this case, the database node needs to load the data to be accessed from the shared data layer in real time, which makes the data access efficiency low and the performance of the database node poor.

[0030] Based on the above technical problems, at least one embodiment of the present specification provides a data caching method, which can be applied to a database node, such as any database node of a database system with a shared storage architecture. When the space used for persistent cache data in the local storage device of the database node shrinks or expands, the method can increase or decrease the persistent cache data accordingly, and in the process of dynamic and elastic changes in the space used for persistent cache data, it can ensure that the data access of the database node has a high hit rate, thereby improving the data service performance of the database node.

[0031] Please refer to the attached Figure 1 , the database system of the shared storage architecture may include a shared data layer and at least one database node communicating therewith, preferably multiple database nodes. The shared data layer may be in the form of object storage, etc., and the shared data layer stores user data and metadata within the system (such as baselines, dumps, logs, and other data) for sharing by multiple database nodes. The database node has a local storage device and a memory. The local storage device is a persistent storage medium, such as a disk, which is used to persistently cache part of the data in the shared storage and hopes that the cached data is data that is subsequently accessed or even frequently accessed by the database node. The memory is a non-persistent storage medium, which is used to store metadata of persistent cached data. The metadata may include a key value, a storage space required size, an offset offset, a region to which it belongs (i.e., in a cache area or a backup cache area), an access heat value, etc.

[0032] Each database node exists as an instance or copy of the data system. The data system forms a storage-computing separation architecture based on shared storage. The data stored in the shared data layer not only avoids the occupation of local storage space of the database node, but also realizes data sharing between different database nodes.

[0033] Please refer to the attached Figure 2 , which exemplarily shows a flow chart of the method, including step S201 and step S202.

[0034] In step S201, if the space occupied by the persistent cached data on the local storage device is greater than the persistent cache threshold, part of the data in the persistent cache on the local storage device is deleted so that the space occupied by the persistent cached data on the local storage device is no greater than the persistent cache threshold, and the metadata of the deleted data is transferred from the in-memory cache area to the backup cache area.

[0035] The persistent cache threshold is a space threshold allocated by the database node on the local storage device for persistent cache.

[0036] Among them, there are cache areas and backup cache areas configured internally, both of which are used to store metadata of data. The cache area is configured with a latest access list and a frequently accessed list, and the metadata in the cache area is divided into the above two lists. If the data to which the metadata belongs is recently accessed, the metadata of the data is divided into the latest access list. If the data to which the metadata belongs is frequently accessed, or is both recently accessed and frequently accessed, the metadata of the data is divided into the frequently access list. The backup cache area is configured with a latest access list and a frequently accessed list, and the metadata in the backup cache area is divided into the above two lists.

[0037] For example, when deleting some data in the persistent cache on the local storage device in this step, some data whose metadata is in the latest access list can be preferentially selected for deletion. For another example, when deleting some data in the persistent cache on the local storage device in this step, some data with a lower access heat value can be preferentially selected for deletion.

[0038] The space occupied by the data in the persistent cache on the local storage device mentioned in this step is greater than the persistent cache threshold, which is often caused by the reduction of the database node, that is, the database node has lowered the persistent cache threshold. This is because the database node will strictly control the space occupied by the data in the persistent cache when performing persistent caching of data, so that it does not exceed the persistent cache threshold. Specifically, the method can control the space occupied by the data in the persistent cache to not exceed the persistent cache threshold in the following cache elimination manner: if the space required for the current data to be cached is greater than the remaining space in the space used for persistent caching on the local storage device, then delete part of the data in the persistent cache on the local storage device to cache the current data to be cached to the local storage device, and transfer the metadata of the deleted data from the cache area in the memory to the backup cache area. In this scenario, part of the data in the persistent cache with a lower heat value can be preferentially selected for deletion. In this scenario, part of the data in the persistent cache whose metadata is in the latest access list can be preferentially selected for deletion.

[0039] It can be seen from the operations in this step that the data to which the metadata in the backup cache area belongs is not persistently cached on the local storage device, that is, the data to which the metadata in the backup cache area belongs is data that the database node hopes to cache persistently to the current storage device but cannot cache due to its limited space. In other words, it is data with high access popularity that is forced to be eliminated.

[0040] For example, in this step, when metadata is transferred from the cache area to the backup cache area, metadata in the latest access list in the cache area can be transferred to the latest access list in the backup cache area, and metadata in the frequently access list in the cache area can be transferred to the frequently access list in the backup cache area.

[0041] For another example, in this step, when the metadata is transferred from the cache area to the backup cache area, if the remaining space in the backup cache area in the memory is less than the space required for the metadata of the deleted data, the space allocated to the backup cache area in the memory is adjusted to be not less than the space required for the metadata of the deleted data, and the metadata of the deleted data is transferred from the cache area to the backup cache area. Since the total amount of metadata remains unchanged during the above transfer process, and the storage space in the memory used to load the metadata has not changed, the backup cache area can be expanded by adjusting the ratio between the cache area and the backup cache area, so that the metadata of the deleted data can be retained in the memory. As mentioned above, the data to which the metadata in the backup cache area belongs is the data that the database node hopes to cache persistently to the current storage device but cannot cache due to its limited space. Through the method of this example, the database node can record as much data as possible that it hopes to cache.

[0042] In step S402, if the space occupied by the persistent cache data on the local storage device is less than the persistent cache threshold, the data belonging to part or all of the metadata in the backup cache area in the memory is cached on the local storage device, and the metadata of the cached data is transferred from the backup cache area in the memory to the cache area.

[0043] As mentioned above, the data to which the metadata in the backup cache area belongs is data that the database node hopes to cache persistently to the current storage device but cannot be cached due to its limited space. Step S401 loads the metadata of this part of the data when the database node is reduced in capacity (i.e., the persistent cache threshold is reduced), so that this step needs to increase the data in the persistent cache with a certain degree of targeting, thereby increasing the access hit rate of the database node, especially compared with the method of randomly selecting data for persistent cache, it overcomes the blindness and increases the access hit rate of the database node.

[0044] Under the architecture of a shared storage system, this step may include: if the space occupied by the persistent cached data on the local storage device is less than the persistent cache threshold, the data belonging to part or all of the metadata in the backup cache area in the memory is cached from the shared data layer to the local storage device, and the metadata of the cached data (i.e., the newly cached data in this step) is transferred from the backup cache area in the memory to the cache area.

[0045] It should be understood that if metadata is stored in the backup cache area in the memory, it means that the database node has previously recorded data in the backup cache area due to shrinkage. If metadata is not stored in the backup cache area in the memory, it means that the database node has not previously recorded data in the backup cache area due to shrinkage.

[0046] The scenario targeted by this step is that the database node is expanded again after a period of time after being reduced in capacity. At this time, the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold because the expansion causes a sudden increase in the persistent cache threshold, and metadata is stored in the backup cache area at this time. Therefore, when caching the data belonging to some or all of the metadata in the backup cache area in the memory to the local storage device, this step can be performed in the manner provided by the following two optional examples.

[0047] Optional Example 1 If the space occupied by the data of the persistent cache on the local storage device is less than the persistent cache threshold, there is metadata in the backup cache area in the memory, and the space required for the data belonging to the metadata in the backup cache area is not greater than the remaining space in the space used for persistent cache on the local storage device, then the data belonging to all the metadata in the backup cache area will be cached to the local storage device.

[0048] Optional Example 2 If the space occupied by the data of the persistent cache on the local storage device is less than the persistent cache threshold, there is metadata in the backup cache area in the memory, and the space required for the data belonging to the metadata in the backup cache area is greater than the remaining space in the space used for persistent cache on the local storage device, then the data belonging to part of the metadata in the backup cache area will be cached to the local storage device.

[0049] For example, in this example, the data to which part of the metadata in the backup cache area belongs may be cached to the local storage device on the principle of preferentially transferring the data to which the metadata in the frequently accessed list belongs.

[0050] For another example, in this example, the data belonging to part of the metadata in the backup cache area may be cached to the local storage device based on the principle of prioritizing the transfer of data with high access popularity.

[0051] If no metadata is stored in the backup cache area in memory, it means that the remaining persistent cache space in the local storage device of the database node is not fully occupied after the database node is running. In this case, you only need to follow the general data persistent cache method, that is, cache part of the data in the shared data layer to the local storage device of the database node to gradually fill up its persistent cache space, and add the metadata of the persistent cache data to the cache area in memory.

[0052] When expanding the capacity of a database node, you can use two methods: vertical expansion and horizontal expansion. Vertical expansion refers to adding local storage devices on a single machine for persistent cache space. Horizontal expansion refers to adding extension nodes to the database node to increase local storage devices, and at the same time using part or all of the space of the newly added local storage devices to increase the space for persistent cache.

[0053] For example, vertical expansion can complete persistent caching of data in the manner described in the above embodiment.

[0054] For another example, horizontal expansion can further complete the persistent caching of data in the following manner: if an extension node is added to the database node so that the space occupied by the persistently cached data on the local storage device of the database node is less than the persistent cache threshold, the metadata in the backup cache area in the memory of the database node is sent to the extension node, so that the extension node caches the data belonging to part or all of the received metadata on the local storage device, adds the metadata of the cached data to the cache area in the memory, and adds the remaining metadata to the backup cache area in the memory.

[0055] Compared with vertical expansion, horizontal expansion requires the original node to share the metadata in the backup cache area in memory to the newly added expansion node, so that the expansion node can persist and cache part of the data in the shared data layer based on the metadata in the backup cache area.

[0056] As for more details involved in horizontal expansion, it can be completed in the manner described in the above embodiment: Cache some data in the shared data layer to local storage devices.

[0057] If the space required for the data to which all metadata received by the extension node belongs is not greater than the space used for persistent cache on the local storage device of the extension node, the data to which all metadata received belongs is cached in the local storage device.

[0058] If the space required for the data belonging to all metadata received by the extension node is greater than the space used for persistent caching on the local storage device of the extension node, the data belonging to part of the received metadata is cached to the local storage device. Preferably, the data belonging to part of the metadata in the backup cache area is cached to the local storage device on the principle of giving priority to transferring the data belonging to the metadata in the frequently accessed list. Preferably, the data belonging to part of the metadata in the backup cache area is cached to the local storage device on the principle of giving priority to transferring the data with high access popularity.

[0059] Please refer to the attached Figure 3 Next, the above embodiments are combined with the obtained data caching method using a specific scenario of an example, which includes the following steps: Persistent cache space allocation: The persistent cache threshold in the local disk of the database node is 100 GB, and the in-memory cache area is the space required for the metadata of 100 GB persistent cache data.

[0060] Data persistent cache: As data is accessed, the database node gradually fills up the space used for persistent cache on the local disk, that is, 100GB of persistent cache data is written.

[0061] Reduction of persistent cache space: When the database node is reduced in size and the persistent cache threshold is reduced to 20 GB, 80 GB of the 100 GB persistent cache data is eliminated, that is, deleted, and the cache area in the memory is configured to the space required for the metadata of the 20 GB persistent cache data, the backup cache area is configured to the space required for the metadata of the 80 GB persistent cache data, and the metadata of the deleted 80 GB data is transferred from the cache area to the backup cache area.

[0062] Persistent cache space expansion: When the database node is expanded and the persistent cache threshold is increased to 100GB, 80GB of data belonging to the metadata in the backup cache area is persistently cached to the local disk to reach the persistent cache threshold. For example, the data to be persistently cached to the local disk is selected by the key in the metadata in the backup cache area.

[0063] In this process, the metadata of the deleted data in the scaling scenario is recorded in the backup cache area of ​​the memory. In the scaling scenario, the data can be persistently cached based on the metadata recorded in the backup cache area, overcoming the blindness and randomness in the related technologies and improving the hit rate of database node access. Moreover, when recording the metadata in the backup cache area, the space of the backup cache area is expanded to record as much metadata as possible, so that the scaling scenario can have enough metadata to guide the selection of data for persistent cache.

[0064] In the data caching method provided in the embodiment of this specification, the database node configures a cache area and a backup cache area in the memory. When the database node persists the cached data, it stores the data on a local storage device such as a disk, and loads the metadata of the cached data into the cache area. When the space used for persistent caching in the local storage device is reduced, part of the data in the persistent cache can be deleted, and the metadata of the deleted data can be transferred from the cache area to the backup cache area. When the space used for persistent caching in the local storage device of the database node is expanded, the data to which the metadata in the backup cache area belongs can be persistently cached to the local storage device, and the metadata of the cached data can be transferred from the backup cache area to the cache area. It can be seen that the data to which the metadata in the backup cache area belongs is the data that the database node wants to cache but cannot cache due to limited space. Recording the metadata of this part of the data can increase the persistent cached data in a targeted manner when the space increases, so that the added data has a higher hit rate, thereby avoiding the problem of low hit rate caused by randomly increasing the persistent cached data when the space increases in the related art. The higher hit rate makes the data service of the database node have lower latency, higher efficiency and better performance.

[0065] Figure 4 is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 4At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410, and may also include hardware required for other tasks. One or more embodiments of this specification may be implemented based on software, such as the processor 402 reading the corresponding computer program from the non-volatile memory 410 into the memory 408 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0066] Please refer to Figure 5 , the data cache device can be used for example Figure 4 The database node running on the device shown in the figure implements the technical solution of this specification. The data cache device may include: The shrinking module 501 is used to delete part of the data in the persistent cache on the local storage device if the space occupied by the data in the persistent cache on the local storage device is greater than the persistent cache threshold, so that the space occupied by the data in the persistent cache on the local storage device is not greater than the persistent cache threshold, and transfer the metadata of the deleted data from the in-memory cache area to the backup cache area; The expansion module 502 is used to cache the data belonging to part or all of the metadata in the backup cache area in the memory on the local storage device if the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold, and transfer the metadata of the cached data from the backup cache area in the memory to the cache area; The persistent cache threshold is a space threshold allocated by the database node on the local storage device for persistent cache.

[0067] In one embodiment of the present specification, the device further comprises an elimination module, which is used to: If the space required for the data to be cached currently is greater than the remaining space in the space used for persistent caching on the local storage device, part of the data in the persistent cache on the local storage device is deleted to cache the data to be cached currently to the local storage device, and the metadata of the deleted data is transferred from the cache area in the memory to the backup cache area.

[0068] In one embodiment of this specification, the expansion module is used to: If the space occupied by the persistent cached data on the local storage device is less than the persistent cache threshold, the data belonging to some or all of the metadata in the backup cache area in the memory will be cached by the shared data layer to the local storage device, and the metadata of the cached data will be transferred from the backup cache area in the memory to the cache area.

[0069] In one embodiment of the present specification, the expansion module is used to cache data belonging to part or all of the metadata in the backup cache area in the memory on the local storage device if the space occupied by the persistent cache data on the local storage device is less than the persistent cache threshold, and is used to: If the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold, there is metadata in the backup cache area in the memory, and the space required for the data belonging to the metadata in the backup cache area is less than the remaining space in the space used for persistent cache on the local storage device, then cache the data belonging to all the metadata in the backup cache area to the local storage device; If the space occupied by the data of the persistent cache on the local storage device is less than the persistent cache threshold, there is metadata in the backup cache area in the memory, and the space required for the data belonging to the metadata in the backup cache area is greater than the remaining space in the space used for persistent cache on the local storage device, then the data belonging to part of the metadata in the backup cache area will be cached to the local storage device.

[0070] In one embodiment of the present specification, the backup cache area includes a latest access list and a frequently accessed list; when the expansion module is used to cache data belonging to part of the metadata in the backup cache area to the local storage device, it is used to: Based on the principle of giving priority to transferring the data belonging to the metadata in the frequently accessed list, the data belonging to part of the metadata in the backup cache area are cached to the local storage device.

[0071] In one embodiment of the present specification, the metadata includes access popularity of the data to which it belongs; when the expansion module is used to cache the data to which part of the metadata in the backup cache area belongs to the local storage device, it is used to: Based on the principle of giving priority to transferring data with high access popularity, data belonging to part of the metadata in the backup cache area is cached to the local storage device.

[0072] In one embodiment of the present specification, when the capacity expansion module is used to transfer metadata of deleted data in the memory from the cache area to the backup cache area, it is used to: If the remaining space in the backup cache area in the memory is less than the space required for the metadata of the deleted data, the space allocated to the backup cache area in the memory is adjusted to be no less than the space required for the metadata of the deleted data, and the metadata of the deleted data is transferred from the cache area to the backup cache area.

[0073] In one embodiment of this specification, the expansion module is used to: If an extension node is added to the database node so that the space occupied by the persistently cached data on the local storage device of the database node is less than the persistent cache threshold, the metadata in the backup cache area in the memory of the database node is sent to the extension node, so that the extension node caches the data belonging to part or all of the received metadata on the local storage device, adds the metadata of the cached data to the cache area in the memory, and adds the remaining metadata to the backup cache area in the memory.

[0074] More details about the various modules in the device have been introduced in detail in the data caching method of the first aspect, and will not be repeated here.

[0075] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0076] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0077] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0078] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0079] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0080] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0081] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0082] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0083] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0084] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A data caching method, applied to a database node, comprising: If the space occupied by the data in the persistent cache on the local storage device is greater than the persistent cache threshold, some of the data in the persistent cache on the local storage device is deleted so that the space occupied by the data in the persistent cache on the local storage device is not greater than the persistent cache threshold, and the metadata of the deleted data is transferred from the in-memory cache area to the backup cache area; If the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold, cache the data belonging to part or all of the metadata in the backup cache area in the memory to the local storage device, and transfer the metadata of the cached data from the backup cache area in the memory to the cache area; The persistent cache threshold is a space threshold allocated by the database node on the local storage device for persistent cache.

2. The data caching method according to claim 1, further comprising: If the space required for the data to be cached currently is greater than the remaining space in the space used for persistent caching on the local storage device, part of the data in the persistent cache on the local storage device is deleted to cache the data to be cached currently to the local storage device, and the metadata of the deleted data is transferred from the cache area in the memory to the backup cache area.

3. The data caching method according to claim 1, wherein if the space occupied by the persistent cached data on the local storage device is less than the persistent cache threshold, the data belonging to part or all of the metadata in the backup cache area in the memory is cached on the local storage device, and the metadata of the cached data is transferred from the backup cache area in the memory to the cache area, comprising: If the space occupied by the persistent cached data on the local storage device is less than the persistent cache threshold, the data belonging to some or all of the metadata in the backup cache area in the memory will be cached by the shared data layer to the local storage device, and the metadata of the cached data will be transferred from the backup cache area in the memory to the cache area.

4. The data caching method according to claim 1, wherein if the space occupied by the persistent cache data on the local storage device is less than the persistent cache threshold, caching the data belonging to part or all of the metadata in the backup cache area in the memory on the local storage device, comprises: If the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold, there is metadata in the backup cache area in the memory, and the space required for the data belonging to the metadata in the backup cache area is less than the remaining space in the space used for persistent cache on the local storage device, then cache the data belonging to all the metadata in the backup cache area to the local storage device; If the space occupied by the data of the persistent cache on the local storage device is less than the persistent cache threshold, there is metadata in the backup cache area in the memory, and the space required for the data belonging to the metadata in the backup cache area is greater than the remaining space in the space used for persistent cache on the local storage device, then the data belonging to part of the metadata in the backup cache area will be cached to the local storage device.

5. The data caching method according to claim 4, wherein the backup cache area includes a latest access list and a frequently accessed list; and caching the data belonging to part of the metadata in the backup cache area to the local storage device comprises: Based on the principle of giving priority to transferring the data belonging to the metadata in the frequently accessed list, the data belonging to part of the metadata in the backup cache area are cached to the local storage device.

6. The data caching method according to claim 4, wherein the metadata includes access popularity of the data to which it belongs; and caching the data to which the metadata in the backup cache area belongs to the local storage device comprises: Based on the principle of giving priority to transferring data with high access popularity, data belonging to part of the metadata in the backup cache area is cached to the local storage device.

7. The data caching method according to claim 1, wherein the step of transferring metadata of the deleted data in the memory from the cache area to the backup cache area comprises: If the remaining space in the backup cache area in the memory is less than the space required for the metadata of the deleted data, the space allocated to the backup cache area in the memory is adjusted to be no less than the space required for the metadata of the deleted data, and the metadata of the deleted data is transferred from the cache area to the backup cache area.

8. The data caching method according to claim 1, wherein if the space occupied by the persistent cached data on the local storage device is less than the persistent cache threshold, the data belonging to part or all of the metadata in the backup cache area in the memory is cached on the local storage device, and the metadata of the cached data is transferred from the backup cache area in the memory to the cache area, comprising: If an extension node is added to the database node so that the space occupied by the persistently cached data on the local storage device of the database node is less than the persistent cache threshold, the metadata in the backup cache area in the memory of the database node is sent to the extension node, so that the extension node caches the data belonging to part or all of the received metadata on the local storage device, adds the metadata of the cached data to the cache area in the memory, and adds the remaining metadata to the backup cache area in the memory.

9. A data cache device, applied to a database node, comprising: A shrinking module is used to delete part of the data in the persistent cache on the local storage device if the space occupied by the data in the persistent cache on the local storage device is greater than the persistent cache threshold, so that the space occupied by the data in the persistent cache on the local storage device is not greater than the persistent cache threshold, and transfer the metadata of the deleted data from the in-memory cache area to the backup cache area; The expansion module is used to cache the data belonging to part or all of the metadata in the backup cache area in the memory on the local storage device if the space occupied by the data in the persistent cache on the local storage device is less than the persistent cache threshold, and transfer the metadata of the cached data from the backup cache area in the memory to the cache area; The persistent cache threshold is a space threshold allocated by the database node on the local storage device for persistent cache.

10. The data cache device according to claim 9, further comprising an elimination module, configured to: If the space required for the data to be cached currently is greater than the remaining space in the space used for persistent caching on the local storage device, part of the data in the persistent cache on the local storage device is deleted to cache the data to be cached currently to the local storage device, and the metadata of the deleted data is transferred from the cache area in the memory to the backup cache area.

11. The data cache device according to claim 9, wherein the backup cache area includes a latest access list and a frequently accessed list; and when the expansion module is used to cache data belonging to part of the metadata in the backup cache area to the local storage device, it is used to: Based on the principle of giving priority to transferring the data belonging to the metadata in the frequently accessed list, the data belonging to part of the metadata in the backup cache area are cached to the local storage device.

12. The data cache device according to claim 9, wherein the metadata includes access popularity of the data to which it belongs; when the expansion module is used to cache the data to which the metadata in the backup cache area belongs to the local storage device, it is used to: Based on the principle of giving priority to transferring data with high access popularity, data belonging to part of the metadata in the backup cache area is cached to the local storage device.

13. The data cache device according to claim 9, wherein when the expansion module is used to transfer metadata of deleted data in the memory from the cache area to the backup cache area, it is used to: If the remaining space in the backup cache area in the memory is less than the space required for the metadata of the deleted data, the space allocated to the backup cache area in the memory is adjusted to be no less than the space required for the metadata of the deleted data, and the metadata of the deleted data is transferred from the cache area to the backup cache area.

14. The data cache device according to claim 9, wherein the capacity expansion module is used for: If an extension node is added to the database node so that the space occupied by the persistently cached data on the local storage device of the database node is less than the persistent cache threshold, the metadata in the backup cache area in the memory of the database node is sent to the extension node, so that the extension node caches the data belonging to part or all of the received metadata on the local storage device, adds the metadata of the cached data to the cache area in the memory, and adds the remaining metadata to the backup cache area in the memory.

15. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

16. An electronic device comprising: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

17. A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images