Data caching method and device, medium, electronic equipment and program product

Abstract

The invention relates to a data caching method and device, a medium, electronic equipment and a program product, and relates to the technical field of data caching. The data with the access frequency not larger than the preset threshold value in the database is cached through the second memory cache component, the access cost of the first memory cache component is smaller than that of the second memory cache component, and the second memory cache component has the capacity control capability, so that the overall access cost of the data can be greatly reduced; the maximization of the overall efficiency is realized, and the capacity control capability can also be realized on the premise of ensuring the efficient access performance of the data. Besides, the hit rate of the data can be greatly improved, and relatively large GC overhead can be avoided, so that the CPU occupation of the target service is greatly reduced, and the overall consumed time of the interface under the action of various factors can be greatly reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data caching, and in particular to a data caching method, device, medium, electronic device and program product. Background Art

[0002] Internet services often maintain a large amount of data to support the operation of Internet services. In order to optimize link time and improve user experience, data in the database is usually cached in the memory of the server corresponding to the Internet service. When the Internet service needs to read data, it can directly read from the server's memory and avoid reading from the database. Therefore, how to optimize the way to cache data locally has become an increasingly important research direction in this field. Summary of the invention

[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides a data caching method, comprising: caching data in a database corresponding to the target service with an access frequency greater than a preset threshold by a first memory cache component; The data in the database whose access frequency is not greater than the preset threshold is cached by a second memory cache component, wherein the access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability.

[0005] In a second aspect, the present disclosure provides a data cache device, comprising: A first cache module is configured to cache data in a database corresponding to the target service with an access frequency greater than a preset threshold through a first memory cache component; The second cache module is configured to cache data in the database whose access frequency is not greater than the preset threshold through a second memory cache component, wherein the access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability.

[0006] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0007] In a fourth aspect, the present disclosure provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0008] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0009] Based on the above technical solution, the first memory cache component is used to cache data in the database corresponding to the target service whose access frequency is greater than a preset threshold, and the second memory cache component is used to cache data in the database whose access frequency is not greater than a preset threshold. The access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability, which can not only significantly reduce the overall access cost of the data and maximize the overall efficiency, but also has capacity control capability while ensuring efficient data access performance. In addition, by combining the first memory cache component with the second memory cache component for data caching, the hit rate of the data can be greatly improved, and a large GC (Garbage Collection) overhead can be avoided, thereby significantly reducing the CPU (Central Processing Unit) occupancy of the target service and significantly reducing the overall time consumption of the interface under the influence of various factors.

[0010] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a flow chart of a data caching method according to some embodiments.

[0012] Figure 2 It is a data flow diagram of a data caching method according to some embodiments.

[0013] Figure 3 is a schematic diagram showing cache inheritance according to some embodiments.

[0014] Figure 4 is a schematic diagram of data query according to some embodiments.

[0015] Figure 5It is a schematic diagram of the structure of a data cache device according to some embodiments.

[0016] Figure 6 is a schematic diagram of the structure of an electronic device according to some embodiments. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0019] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] Figure 1 FIG. 1 is a flow chart of a data caching method according to some embodiments. Figure 1As shown, the embodiment of the present disclosure provides a data caching method, which can be executed by an electronic device, specifically by a data caching device, which can be implemented by software and / or hardware and configured in the electronic device. Figure 1 As shown, the method may include the following steps.

[0024] In step 110, data in a database corresponding to a target service and having an access frequency greater than a preset threshold is cached by a first memory cache component.

[0025] Here, the target service may be an Internet service or an application, and the database corresponding to the target service may be a database for storing data that the target service depends on. Of course, in some scenarios, the data corresponding to the target service may also be a database for storing a specific type of data that the target service depends on. For example, in a payment service, the database corresponding to the payment service may be a database for storing configuration data. In other words, the database corresponding to the target service may refer to a database corresponding to any data that needs to be cached locally in the electronic device.

[0026] Configuration data can be data used to describe marketing activities. For example, if a marketing activity is "From August 16 to August 18, new customers who use bank cards to pay can enjoy two 10 yuan discounts when using bank cards from any channel to pay, with a total budget of 10 million", then the above marketing activity can be described by a configuration data, which includes dozens of fields such as unique identifier (ID), validity period, attributes, budget, and activity rules.

[0027] Configuration data is important data for marketing activities and is used very frequently. In order to reduce the amount of database access, the configuration data in the database can be cached in the local memory of the electronic device.

[0028] In the disclosed embodiment, data in the database corresponding to the target service with an access frequency greater than a preset threshold can be cached in a first memory cache component of the electronic device. It should be understood that the memory cache component is a software component that uses the memory (RAM) of the electronic device to temporarily store data so that the target service can quickly access and retrieve it.

[0029] The preset threshold value may be a frequency value set according to demand. It should be noted that data with an access frequency greater than the preset threshold value may be understood as data that is frequently used in the database.

[0030] It should be understood that in the embodiments of the present disclosure, the specific type of the database is not limited. For example, the database may be MySQL (a widely used open source relational database management system).

[0031] In step 120, data in the database whose access frequency is not greater than a preset threshold is cached by a second memory cache component, wherein the access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability.

[0032] Here, data whose access frequency is not greater than (including greater than or equal to) a preset threshold value can be understood as data in the database that is used infrequently. In the disclosed embodiment, data in the database whose access frequency is not greater than the preset threshold value is cached in the memory of the electronic device through the second memory cache component.

[0033] It should be noted that step 110 and step 120 may be executed simultaneously, or step 110 may be executed first and then step 120, or step 110 may be executed first and then step 120. That is, step 110 and step 120 may be executed in any order.

[0034] In an embodiment of the present disclosure, a first memory cache component is used to cache high-frequency data in a database that is frequently used by a target service (data with an access frequency greater than a preset threshold), and a second memory cache component is used to cache low-frequency data in a database that is infrequently used by a target service (data with an access frequency not greater than a preset threshold).

[0035] It is worth noting that the access cost of the first memory cache component is less than the access cost of the second memory cache component. Among them, the access cost mainly includes the time consumed to access data and the CPU (Central Processing Unit) consumption. Since the first memory cache component is used to cache data with an access frequency greater than a preset threshold, the data in the first memory cache component will be used frequently by the target service. Using a memory cache component with a lower access cost can improve the performance of the target service in accessing high-frequency data. Since the second memory cache component is used to cache data with an access frequency not greater than a preset threshold, the data in the second memory cache component will not be used frequently by the target service. The second memory cache component has a capacity control capability and can evict some low-frequency data from the second memory cache component after the memory capacity reaches the limit to avoid exceeding the memory capacity limit.

[0036] That is, the first memory cache component and the second memory cache component can be memory cache components with complementary performance. The first memory cache component is used to provide high access performance, and the second memory cache component is used to provide capacity control capabilities to adapt to the local cache of ultra-large-scale data, thereby taking into account both capacity control and low-cost access in the local cache of ultra-large-scale data.

[0037] The first memory cache component and the second memory cache component may adopt different data storage architectures. In some embodiments, the first memory cache component may be a memory cache component that adopts an object storage data storage architecture. The second memory cache component may be a memory cache component that adopts a byte storage data storage architecture.

[0038] Among them, the memory cache component of the data storage architecture using object storage generally has higher access performance, but when the amount of data is large, it is easy to exceed the upper limit of the memory capacity. The memory cache component of the data storage architecture using byte storage is more functional and often has capacity control capabilities, but has lower read performance and higher access costs.

[0039] In an embodiment of the present disclosure, a first memory cache component of a data storage architecture using object storage and a second memory cache component of a data storage architecture using byte storage are combined for local caching, so that the overall efficiency of the local cache is maximized.

[0040] For example, in payment services, the amount of data cached locally is often large (up to 10GB), and the data access volume is also high. In order to adapt to the ultra-large-scale local cache of payment services, by using a first memory cache component that uses an object storage data storage architecture and a second memory cache component that uses a byte storage data storage architecture, low-cost access and capacity control can be taken into account at the same time to maximize the overall efficiency of the local cache.

[0041] Exemplarily, the first memory cache component may be SyncMap, and the second memory cache component may be FreeCache.

[0042] SyncMap is a component in the Go language standard library that implements local caching. SyncMap supports concurrent reading and writing, and is suitable for scenarios with more reading and less writing. In addition, SyncMap uses an object storage data storage architecture, which can provide very high access performance. FreeCache is a high-performance Go language memory cache library that supports zero GC (Garbage Collection) overhead and high concurrency performance. Moreover, FreeCache has the LRU (Least Recently Used) feature, which can provide capacity control capabilities.

[0043] In the disclosed embodiment, SyncMap is used to cache data in the database whose access frequency is greater than a preset threshold, and FreeCache is used to cache data in the database whose access frequency is greater than a preset threshold. SyncMap can be used to ensure the access performance of high-frequency data to support the needs of online traffic, but the maintenance cost of SyncMap is high and the capacity cannot be too large, so FreeCache is used to cache low-frequency data, and the excellent capacity control capability of FreeCache can be used to control the capacity of the cache. Moreover, the LRU elimination strategy provided by FreeCache can also ensure that data with a high probability of use can remain in the memory. Therefore, through the caching method that combines SyncMap and FreeCache, the data with the highest probability of use by the target service can be cached in the memory when the memory capacity is limited, and the overall access efficiency of the data can be improved. In addition, by using FreeCache, large GC overhead can also be avoided, so that the CPU usage of the target service is greatly reduced, and the overall time consumption of the interface under the influence of various factors can also be greatly reduced.

[0044] Thus, the first memory cache component caches data in the database corresponding to the target service whose access frequency is greater than a preset threshold, and the second memory cache component caches data in the database whose access frequency is not greater than a preset threshold. The access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability, which can not only significantly reduce the overall access cost of the data and maximize the overall efficiency, but also has capacity control capability while ensuring efficient data access performance. In addition, by combining the first memory cache component with the second memory cache component for data caching, the hit rate of the data can be significantly improved, and a large GC overhead can be avoided, thereby significantly reducing the CPU usage of the target service and significantly reducing the overall time consumption of the interface under the influence of various factors.

[0045] In some feasible implementations, the number of data accesses corresponding to each data in the second memory cache component may be determined, and data with a data access number greater than a preset threshold may be transferred to the first memory cache component.

[0046] Here, the number of data accesses corresponding to each data in the second memory cache component can be determined by a memory counter. The memory counter can count the number of data accesses to the data based on locks and maps, and the memory counter is used to mark the number of data accesses to each data in the second memory cache component. When the data in the second memory cache component is used by the target service, the number of data accesses corresponding to the memory counter is increased by 1.

[0047] It should be understood that since the second memory cache component is used to cache data whose access frequency is not greater than the preset threshold, and the access cost of the second memory cache component is greater than the access cost of the first memory cache component, in order to further improve the overall access efficiency, data whose access frequency is greater than the preset threshold can be transferred to the first memory cache component.

[0048] Among them, data with a data access number greater than the preset threshold can be understood as high-frequency data in the second memory cache component, which is accessed more frequently by the target service. Then, data with a data access number greater than the preset threshold can be transferred from the second memory cache component to the first memory cache component to improve the access efficiency of high-frequency data.

[0049] Exemplarily, at intervals of a first preset time duration, the number of data accesses corresponding to each data in the second memory cache component recorded by the memory counter within the first preset time duration can be obtained, and then the data with a data access number greater than a preset threshold can be transferred to the first memory cache component.

[0050] After the target service is started, a resident coroutine can be created. The resident coroutine triggers a task once every first preset time interval. In response to the task, the number of data access times is pulled from the memory counter, and then for each pulled data access number, it is determined whether the data access number is greater than the preset threshold. If it is greater than the preset threshold, the corresponding data is transferred to the first memory cache component. If it is not greater than the preset threshold, no processing is performed, and the corresponding data continues to be cached in the second memory cache component.

[0051] It is worth noting that the first preset duration can be set according to demand, for example, it can be set to 2 minutes. Of course, the first preset duration can also be other values. The preset threshold can be set by the user according to demand. Of course, the preset threshold can also be dynamically changed, and the preset threshold can be adjusted according to the traffic information of the target service and the response information of each interface in the target service to the traffic information, so that the adjusted preset threshold can make the performance indicator of the target service meet the demand.

[0052] Therefore, by transferring data in the second memory cache component whose access times are greater than a preset threshold to the first memory cache component, a small amount of capacity can be exchanged for greater access performance, achieving the most efficient distribution of data and improving the overall efficiency of the cache.

[0053] In some feasible implementations, the number of data accesses corresponding to each data in the second memory cache component may be reset periodically.

[0054] Here, the popularity of data itself changes dynamically over time. The popularity of the same data will be different at different time nodes. By periodically resetting the data access counts corresponding to each data in the second memory cache component, the data access counts can be counted according to the real-time popularity of the data, and then the data to be transferred to the first memory cache component can be determined according to the real-time popularity of the data to ensure that the target service can make an accurate response.

[0055] Exemplarily, periodically resetting the number of data accesses may be resetting the number of data accesses at intervals of a second preset time length, wherein the second preset time length may be set as required, for example, the second preset time length may be set to 2 minutes.

[0056] Therefore, by periodically resetting the number of data accesses, not only can the infinite expansion of the count map of the memory counter be avoided, but also it is possible to decide which data to transfer to the first memory cache component based on the real-time popularity of the data to ensure that the target service can make an accurate response.

[0057] In some achievable implementations, in response to starting the target service, data of a preset type in a database corresponding to the target service may be cached in the first memory cache component as data with an access frequency greater than a preset threshold value in step 110. Accordingly, in step 120, data in the database that does not belong to the preset type may be cached in the second memory cache component as data with an access frequency not greater than a preset threshold value.

[0058] Here, starting the target service may refer to starting the loading of the target service. Since the number of data accesses to each data in the database is generally not maintained, when starting the target service, the data in the database that belongs to the preset type may be directly used as data with an access frequency greater than a preset threshold, and the data in the database that does not belong to the preset type may be used as data with an access frequency not greater than a preset threshold. That is, when the target service is started, the data in the database of the target service that belongs to the preset type may be cached in the first memory cache component, and the data in the database of the target service that does not belong to the preset type may be cached in the second memory cache component.

[0059] It should be understood that the preset type can be set by the user according to actual conditions. In different scenarios, the user can select which type of data is data with an access frequency greater than a preset threshold.

[0060] For example, when analyzing the 150,000 configuration data of the above-mentioned payment service, it was found that there were 2,000 manually created configuration data and 148,000 automatically created configuration data, and the access frequency of manually created configuration data was greater than that of automatically created configuration data. Therefore, the non-automatically created configuration data (manually created configuration data) can be set as a preset type, that is, the manually created configuration data in the database corresponding to the payment service can be cached in the first memory cache component, and the automatically created configuration data can be cached in the second memory cache component.

[0061] It is worth noting that after the service is started, the data with a data access number greater than a preset threshold in the second memory cache component can be transferred to the first memory cache component through the data access number of the above embodiment to make the overall access efficiency of the cache higher.

[0062] Therefore, through the above implementation, when the target service is started, data with an access frequency greater than a preset threshold can be cached in the first memory cache component, and data with an access frequency not greater than the preset threshold can be cached in the second memory cache component, thereby realizing hierarchical caching of data to improve the overall access efficiency of the cache.

[0063] Figure 2 FIG. 4 is a data flow diagram of a data caching method according to some embodiments. Figure 2 As shown, when the target service is started, the data of the preset type in the database is loaded into the first memory cache component, and the data of the database that does not belong to the preset type is loaded into the second memory cache component by cache inheritance. For data that has been incrementally refreshed in the database, it will be updated to the corresponding memory cache component. Among them, incremental refresh means that the original data in the database is modified, and the modified data will be synchronized to the corresponding first memory cache component or second memory cache component. When the second memory cache component needs to read data back to the source, the data read back to the source will be written back to the second memory cache component. In addition, the number of data accesses corresponding to the data in the second memory cache component can be counted by a memory counter to transfer data whose data access times are greater than a preset threshold to the first memory cache component.

[0064] In some feasible implementations, in step 120, in response to starting the target service, a cache inheritance file can be obtained from the file system, and then corresponding data can be read from the database based on the data identifier in the cache inheritance file, and the read data can be written to the second memory cache component corresponding to the target service.

[0065] Here, at the initial stage of the target service startup, if the data in the database with an access frequency not exceeding a preset threshold is cached in the second memory cache component by relying on the back-to-source method, it may take a long time to load most of the data that needs to be used into the second memory cache component. However, during this period of time at the initial startup, if the data that the target service needs to use is in the database, it is necessary to access the database, which will inevitably lead to a large negative impact on the performance and time consumption of the target service. In order to be able to quickly cache data with an access frequency not exceeding a preset threshold in the second memory cache component at the initial stage of startup, so that the second memory cache component can quickly reach a highly available state, the cache inheritance file can be obtained from the file system, and data with an access frequency not exceeding the preset threshold can be quickly written to the second memory cache component through cache inheritance.

[0066] Among them, the file system can be a TOS file system located at a remote end. For all services of the same type as the target service (including the target service itself), each service has a corresponding first memory cache component and a second memory cache component, and each service can store the data identifier corresponding to the data cached in the second memory cache component corresponding to the service in the cache inheritance file of the file system. When a new service (target service) is started, the data cached in the second memory cache component corresponding to the old service (the service of the same type as the target service) of the same type as the new service can be directly inherited to the second memory cache component of the new service through the data identifier in the cache inheritance file.

[0067] In some embodiments, data identifiers corresponding to data cached in a second memory cache component corresponding to the target service and used by the target service within a preset time period may be periodically obtained, and then the data identifiers may be updated to the cache inheritance file.

[0068] Here, the preset duration can be set by the user according to the needs, for example, the preset duration can be set to 2 minutes. After the target service is started, the target service can periodically obtain the data identifier corresponding to the data cached in the second memory cache component corresponding to the target service that has been used by the target service within the preset duration.

[0069] It should be understood that the memory counter can be used to determine whether the data cached in the second memory cache component has been used by the target service within the preset time. For example, if the memory counter records the data used in the past 2 minutes, the data is the data used by the target service within the preset time.

[0070] The electronic device can directly update the acquired data identifier to the cache inheritance file of the file system through the target service. Of course, in order to avoid duplication of data identifiers in the cache inheritance file, when updating the data identifier to the cache inheritance file, the data identifier written to the cache inheritance file within the target duration can be pulled from the file system, and then the data identifier pulled from the file system and the data identifier corresponding to the data cached in the second memory cache component corresponding to the target service that has been used within the preset duration are merged and deduplicated to obtain the deduplicated data identifier, and then the deduplicated data identifier is updated to the cache inheritance file. Among them, the target duration can be set according to demand, for example, the target duration can be set to within 1 hour.

[0071] It is worth noting that since each service will periodically upload to the file system the data identifiers corresponding to the data used by the service within a preset period of time, from an overall perspective, the cache inheritance file of the file system can almost contain the collection of data cached in the corresponding second memory cache component of all services of the same type. When a new service is started, it inherits the data of the old service through the cache inheritance file, so that the second memory cache component of the new service can have the same cache hit rate as the old service online.

[0072] Therefore, through the above implementation, writing data to the second memory cache component based on cache inheritance can solve the problem of low cache hit rate in the early stage of target service startup, thereby ensuring service performance and time consumption.

[0073] Figure 3 FIG. 4 is a schematic diagram of cache inheritance according to some embodiments. Figure 3 As shown, the cache inheritance component can be initialized. When the cache inheritance component starts inheritance, the cache inheritance component generates an initial file, pulls the data identifier in the cache inheritance file from the file system, and writes the pulled data identifier into the initial file. Then, based on the data identifier in the initial file, data is read in pages in the database, and the read data is written into the corresponding second memory cache component.

[0074] When the cache inheritance component starts pushing, the cache inheritance component starts the cache inheritance push coroutine, the cache inheritance push coroutine grabs the lock, and then pulls the data identifiers within the target duration from the file system, and pulls the data identifiers of the data used within the preset duration from the memory counter, and then merges and deduplicates the pulled data, and writes the merged and deduplicated data into the cache inheritance file of the file system.

[0075] In some feasible implementations, in response to a data query instruction, data corresponding to the data identifier can be queried in a first memory cache component based on a data identifier carried in the data query instruction. When the first memory cache component fails to find the data corresponding to the data identifier, data corresponding to the data identifier can be queried in a second memory cache component based on the data identifier. When the second memory cache component fails to find the data corresponding to the data identifier, data corresponding to the data identifier can be queried in a database based on the data identifier.

[0076] Here, the data query instruction may be triggered by the target service, and when the target service needs to use a piece of data, the target service may query based on the data identifier, wherein the data identifier refers to a unique identifier of the data.

[0077] Figure 4 FIG. 1 is a schematic diagram of data query according to some embodiments. Figure 4 As shown, the target service can query the first memory cache component based on the data identifier. If the corresponding data is not found in the first memory cache component, the corresponding data is queried in the second memory cache component based on the data identifier. If the corresponding data is found in the second memory cache component, the queried data is deserialized to obtain the data required by the target service. If the corresponding data is not found in the second memory cache component, the corresponding data is queried in the database based on the data identifier.

[0078] It should be noted that the data queried in the database can also be updated to the second memory cache component by write-back cache to increase the hit rate of the second memory cache component. Of course, if the data in the second memory cache component is used, the data access count will be updated asynchronously and the consistency check will be performed.

[0079] The above implementation is described in detail below through some examples.

[0080] Taking the example of SyncMap as the first memory cache component and FreeCache as the second memory cache component, in the payment service, SyncMap can be used as the local first-level cache and FreeCache as the local second-level cache. By storing high-frequency data in SyncMap, the access performance of high-frequency data is improved to meet the needs of online traffic. By storing low-frequency data in FreeCache, it can have excellent capacity control capabilities and LRU elimination strategies. SyncMap and FreeCache are superimposed to cache data. When capacity is limited, the data with the highest probability of use can be stored in memory as much as possible, and the overall access efficiency of the cache can be maximized.

[0081] On the basis of using SyncMap as the local first-level cache and FreeCache as the local second-level cache, the data with data access times greater than the preset threshold in FreeCache is promptly transferred to SyncMap according to the data access times, which can further improve the overall access efficiency of the cache and enable the use of ultra-large-scale data caching scenarios.

[0082] Moreover, when the payment service is started, cache inheritance can be used to enable the newly started payment service to quickly reach a high availability state, solving the problem of low cache hit rate in the initial stage of payment service startup.

[0083] It is worth noting that in the above examples, although SyncMap and FreeCache are used as examples, it does not mean that the first memory cache component is limited to SyncMap and the second memory cache component is FreeCache. Other memory cache components of data storage architectures that can adopt object storage can all be used as the first memory cache component, and other memory cache components of data storage architectures that can adopt byte storage can all be used as the second memory cache component.

[0084] Figure 5 FIG. 1 is a schematic diagram showing the structure of a data cache device according to some embodiments. Figure 5 As shown, the embodiment of the present disclosure provides a data cache device 500, and the data cache device 500 includes: The first cache module 501 is configured to cache data in a database corresponding to the target service with an access frequency greater than a preset threshold through a first memory cache component; The second cache module 502 is configured to cache data in the database whose access frequency is not greater than the preset threshold through a second memory cache component, wherein the access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability.

[0085] Optionally, the data cache device 500 further includes: A determination module, configured to determine the number of data accesses corresponding to each data in the second memory cache component; The transfer module is configured to transfer the data whose data access times are greater than the preset threshold to the first memory cache component.

[0086] Optionally, the data cache device 500 further includes: The reset module is configured to periodically reset the data access times corresponding to each data in the second memory cache component.

[0087] Optionally, the first cache module 501 is specifically configured as follows: In response to starting the target service, caching the data of a preset type in the database corresponding to the target service as the data with an access frequency greater than a preset threshold in the first memory cache component; The second cache module 502 is specifically configured as follows: The data in the database that does not belong to the preset type is cached in the second memory cache component as data with an access frequency not greater than the preset threshold.

[0088] Optionally, the second cache module 502 is specifically configured as follows: In response to starting the target service, obtaining a cache inheritance file from the file system, the cache inheritance file being used to store data identifiers corresponding to data cached in a second memory cache component corresponding to the target service by a service of the same type as the target service; According to the data identifier in the cache inheritance file, the corresponding data is read from the database, and the read data is written into the second memory cache component corresponding to the target service.

[0089] Optionally, the data cache device 500 further includes: An acquisition module is configured to periodically acquire a data identifier corresponding to data cached in a second memory cache component corresponding to the target service and used by the target service within a preset time period; An update module is configured to update the data identifier into the cache inheritance file.

[0090] Optionally, the data cache device 500 further includes: A first query module is configured to respond to a data query instruction and query the first memory cache component for data corresponding to the data identifier according to the data identifier carried by the data query instruction; A second query module is configured to query the data corresponding to the data identifier in the second memory cache component according to the data identifier when the first memory cache component fails to query the data corresponding to the data identifier; The third query module is configured to query the database for the data corresponding to the data identifier based on the data identifier when the second memory cache component fails to query the data corresponding to the data identifier.

[0091] Optionally, the first memory cache component is a memory cache component of a data storage architecture that adopts object storage, and the second memory cache component is a memory cache component of a data storage architecture that adopts byte storage.

[0092] The logic of the method executed by each functional module in the above-mentioned data cache device 500 can refer to the part of the method related to the above-mentioned embodiment, and will not be repeated here.

[0093] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device (such as a terminal device or a server) 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0094] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0095] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0096] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0097] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0098] In some embodiments, the electronic device and the file system can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0099] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0100] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: caches data in the database corresponding to the target service whose access frequency is greater than a preset threshold through a first memory cache component; and caches data in the database whose access frequency is not greater than the preset threshold through a second memory cache component, wherein the access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability.

[0101] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a module does not, in some cases, limit the module itself.

[0104] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0105] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0107] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0108] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A data caching method, characterized in that: include: caching data in a database corresponding to the target service with an access frequency greater than a preset threshold by a first memory cache component; The data in the database whose access frequency is not greater than the preset threshold is cached by a second memory cache component, wherein the access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability.

2. The method according to claim 1, characterized in that The method further comprises: Determine the number of data accesses corresponding to each data in the second memory cache component; The data whose access times are greater than the preset threshold is transferred to the first memory cache component.

3. The method according to claim 2, characterized in that The method further comprises: The data access times corresponding to each data in the second memory cache component are reset periodically.

4. The method according to claim 1, characterized in that: The step of caching data in a database corresponding to the target service with an access frequency greater than a preset threshold by the first memory cache component includes: In response to starting the target service, caching the data of a preset type in the database corresponding to the target service as the data with an access frequency greater than a preset threshold in the first memory cache component; The step of caching the data in the database with an access frequency no greater than the preset threshold by the second memory cache component includes: The data in the database that does not belong to the preset type is cached in the second memory cache component as data with an access frequency not greater than the preset threshold.

5. The method according to claim 1, characterized in that The step of caching the data in the database with an access frequency no greater than the preset threshold by the second memory cache component includes: In response to starting the target service, obtaining a cache inheritance file from the file system, the cache inheritance file being used to store data identifiers corresponding to data cached in a second memory cache component corresponding to the target service by a service of the same type as the target service; According to the data identifier in the cache inheritance file, the corresponding data is read from the database, and the read data is written into the second memory cache component corresponding to the target service.

6. The method according to claim 5, characterized in that The method further comprises: Periodically obtain data identifiers corresponding to data cached in a second memory cache component corresponding to the target service and used by the target service within a preset time period; The data identifier is updated into the cache inheritance file.

7. The method according to claim 1, characterized in that The method further comprises: In response to the data query instruction, query the first memory cache component for data corresponding to the data identifier according to the data identifier carried by the data query instruction; When the first memory cache component fails to find the data corresponding to the data identifier, querying the second memory cache component for the data corresponding to the data identifier according to the data identifier; When the second memory cache component fails to find the data corresponding to the data identifier, the database is searched for the data corresponding to the data identifier according to the data identifier.

8. The method according to any one of claims 1 to 7, characterized in that The first memory cache component is a memory cache component of a data storage architecture that adopts object storage, and the second memory cache component is a memory cache component of a data storage architecture that adopts byte storage.

9. A data cache device, characterized in that: include: A first cache module is configured to cache data in a database corresponding to the target service with an access frequency greater than a preset threshold through a first memory cache component; The second cache module is configured to cache data in the database whose access frequency is not greater than the preset threshold through a second memory cache component, wherein the access cost of the first memory cache component is less than the access cost of the second memory cache component, and the second memory cache component has capacity control capability.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images