Cache consistency management system and method and electronic equipment

Abstract

The invention relates to a cache consistency management system and method and electronic equipment, and belongs to the technical field of computers. The system comprises a database, a cache node and a server, wherein data stored in the database is consistent with data cached in the cache node; and the server is used for monitoring the activity frequency of the target data in the database or the cache node, and adjusting the current cache failure time of the target data in the cache node according to the activity frequency of the target data. The current cache failure time of the target data in the cache node is automatically adjusted by monitoring the activity frequency of the target data in the database or the cache node, the cache hit rate, the cache space utilization rate and the system management efficiency are improved, and the requirement for manual intervention is reduced.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a cache consistency management system, method and electronic device. Background Art

[0002] In Python backend development, the cache mechanism is an important technical means to improve system performance. The cache invalidation strategy is an important part of the cache mechanism, which determines when and how to remove data from the cache to ensure data consistency. However, the common cache invalidation strategy currently requires manual configuration and tuning, and has problems such as poor adaptability and low management efficiency. Summary of the invention

[0003] In view of this, the purpose of the present application is to provide a cache consistency management system, method and electronic device to improve the current cache invalidation strategy that requires manual configuration and tuning, has poor adaptability and low management efficiency.

[0004] The embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a cache consistency management system, including: a database, a cache node, wherein the data stored in the database is consistent with the data cached in the cache node; a server, for monitoring the activity frequency of target data in the database or the cache node, and adjusting the current cache expiration time of the target data in the cache node according to the activity frequency of the target data.

[0006] In the above embodiment, the activity frequency of the target data in the database or cache node is monitored by the server, and the current cache expiration time of the target data in the cache node is adjusted according to the activity frequency of the target data; thereby providing a new cache consistency management system that can automatically adjust the cache expiration time, which can not only adapt to changing needs, but also effectively determine which data should be retained in the cache, thereby improving the cache hit rate. In addition, the cache consistency management system has a high degree of automation, reduces the need for manual intervention, improves the management efficiency of the system, simplifies the management and maintenance of the cache system, and reduces maintenance costs.

[0007] In combination with the embodiment of the first aspect, in a possible implementation manner, the activity frequency includes a data change frequency, and the server is used to monitor the data change frequency of the target data stored in the database; or, the activity frequency includes a data access frequency, and the server is used to monitor the data access frequency of the target data cached in the cache node.

[0008] In the above embodiment, the data change frequency of the target data stored in the server monitoring database, or the data access frequency of the target data cached in the cache node, is used to adjust the current cache expiration time of the target data in the cache node; by considering multiple dimensions to make cache decisions, the importance of data and cache requirements can be evaluated more comprehensively and accurately, thereby improving the utilization of cache space and the cache hit rate. It can also be dynamically adjusted according to the current system status and external conditions, so that the system can adapt to different workloads and usage patterns, and has strong adaptability.

[0009] In combination with the embodiment of the first aspect, in a possible implementation manner, the activity frequency includes a data change frequency, and the server is used to record the number of changes of the target data stored in the database; and count the number of changes of the target data within a preset time window; and determine the data change frequency of the target data based on the number of changes.

[0010] In the above embodiment, the server records the number of changes to the target data stored in the database, and counts the number of changes to the target data within a preset time window, and determines the data change frequency of the target data based on the number of changes; determining the data change frequency by recording the number of changes to the target data stored in the database can ensure that the data cached in the cache is synchronized with the data stored in the database, reducing unnecessary data synchronization and updates, thereby improving system performance.

[0011] In combination with the first aspect embodiment, in a possible implementation manner, the activity frequency includes a data change frequency, and the server is used to record the timestamp of the target data change stored in the database; and count the number of timestamps of the target data change within a preset time window; and determine the data change frequency of the target data based on the number of timestamps.

[0012] In the above embodiment, the server records the timestamps of target data changes stored in the database, and counts the number of timestamps of target data changes within a preset time window, and determines the data change frequency of the target data based on the number of timestamps. By recording the timestamps of target data changes stored in the database to determine the data change frequency, it can be ensured that the data cached in the cache is synchronized with the data stored in the database, reducing unnecessary data synchronization and updates, thereby improving system performance.

[0013] In combination with the embodiment of the first aspect, in a possible implementation manner, the activity frequency includes a data access frequency, and the server is used to record the number of times the target data cached in the cache node is accessed; and count the number of times the target data is accessed within a preset time window; and determine the data access frequency of the target data based on the number of accesses.

[0014] In the above embodiment, the server records the number of accesses to the target data cached in the cache node, and counts the number of accesses to the target data within a preset time window, and determines the data access frequency of the target data based on the number of accesses. By recording the number of accesses to the target data cached in the cache node to determine the data access frequency, it is possible to ensure that the data cached in the cache is synchronized with the data stored in the database, thereby reducing unnecessary data synchronization and updates, thereby improving system performance.

[0015] In combination with the embodiment of the first aspect, in a possible implementation, the activity frequency includes a data access frequency, and the server is used to record the timestamp of the target data access cached in the cache node; and count the number of timestamps of the target data access within a preset time window; and determine the data access frequency of the target data based on the number of timestamps.

[0016] In the above embodiment, the server records the timestamps of target data accesses cached in the cache node, and counts the number of timestamps of target data accesses within a preset time window, and determines the data access frequency of the target data based on the number of timestamps. By recording the timestamps of target data accesses cached in the cache node to determine the data access frequency, it can be ensured that the data cached in the cache is synchronized with the data stored in the database, reducing unnecessary data synchronization and updates, thereby improving system performance.

[0017] In combination with the first aspect embodiment, in a possible implementation, the activity frequency includes a data change frequency, and the server is used to determine whether the data change frequency of the target data is greater than or equal to a first threshold. If so, the current cache expiration time of the target data is reduced; if not, the current cache expiration time of the target data is extended.

[0018] In the above embodiment, the server determines whether the data change frequency of the target data is greater than or equal to the first threshold. When the data change frequency is greater than or equal to the first threshold, the current cache expiration time of the target data is reduced. When the data change frequency is less than the first threshold, the current cache expiration time of the target data is extended. The current cache expiration time of the target data is automatically adjusted according to the data change situation, which improves the efficiency of the cache, ensures the consistency of the data cached in the cache and the data in the database, and reduces costs.

[0019] In combination with the first aspect embodiment, in a possible implementation, the activity frequency includes a data access frequency, and the server is used to determine whether the data access frequency of the target data is greater than or equal to a second threshold. If it is yes, the current cache expiration time of the target data is extended; if it is no, the current cache expiration time of the target data is reduced.

[0020] In the above embodiment, the server determines whether the data access frequency of the target data is greater than or equal to the second threshold. When the data access frequency is greater than or equal to the second threshold, the current cache expiration time of the target data is extended. When the data access frequency is less than the second threshold, the current cache expiration time of the target data is reduced. The current cache expiration time of the target data is automatically adjusted according to the data access situation, which improves the efficiency of the cache, ensures the consistency of the data cached in the cache and the data in the database, and reduces the cost.

[0021] In a second aspect, an embodiment of the present application provides a cache consistency management method, which is applied to a backend service, wherein the backend service is connected to a database and a cache node respectively, and comprises: monitoring the activity frequency of target data in the database or the cache node, wherein the data stored in the database is consistent with the data cached in the cache node; and adjusting the current cache expiration time of the target data in the cache node according to the activity frequency of the target data.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is connected to the processor; the memory is used to store programs; and the processor is used to call the programs stored in the memory to execute the method provided in the embodiment of the second aspect above.

[0023] Other features and advantages of the present application will be described in the following description. The purpose and other advantages of the present application can be realized and obtained through the structures specifically pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The above and other purposes, features and advantages of the present application will be more clear as shown in the drawings.

[0025] Figure 1 A schematic diagram of the structure of a cache consistency management system provided in an embodiment of the present application is shown.

[0026] Figure 2 A flow chart of a cache consistency management method provided in an embodiment of the present application is shown.

[0027] Figure 3 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The following embodiments can be used as examples to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the scope of protection of the present application. It will be appreciated by those skilled in the art that, in the absence of conflict, the features in the following embodiments and embodiments can be combined with each other.

[0029] It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. Moreover, the terms "comprises," "comprising," or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0030] Furthermore, the term "and / or" in this application is merely a term used to describe the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0031] The embodiment of the present application provides a cache consistency management system, which monitors the activity frequency of target data in a database or cache node through a server, and adjusts the current cache expiration time of the target data in the cache node according to the activity frequency of the target data, and makes cache decisions by considering multiple dimensions. It can evaluate the importance of data and cache requirements more comprehensively and accurately, thereby improving the utilization rate of cache space and the cache hit rate. It can also be dynamically adjusted according to the current system status and external conditions, so that the system can adapt to different workloads and usage patterns, and has strong adaptability. In addition, it has a high degree of automation, reduces the need for manual intervention, simplifies the management and maintenance of the cache system, and reduces maintenance costs.

[0032] like Figure 1 As shown, Figure 1 A schematic diagram of the structure of a cache consistency management system provided in an embodiment of the present application is shown.

[0033] The cache consistency management system includes: a database, a cache node, and a server. The data stored in the database is consistent with the data cached in the cache node. The server is used to monitor the activity frequency of the target data in the database or cache node, and adjust the current cache expiration time of the target data in the cache node according to the activity frequency of the target data.

[0034] In a possible implementation, the activity frequency includes a data change frequency, and the server is used to monitor the data change frequency of the target data stored in the database; or, the activity frequency includes a data access frequency, and the server is used to monitor the data access frequency of the target data cached in the cache node.

[0035] In one implementation, the target data stored in the database may be changed in two ways: content change and type change. Content change refers to a situation where the data type does not change but the value of the data changes; while type change refers to a situation where the data type changes.

[0036] In a possible implementation, the activity frequency includes a data change frequency and a data access frequency, and the server is used to monitor the data change frequency of the target data stored in the database and the data access frequency of the target data cached in the cache node.

[0037] In one possible implementation, when the activity frequency includes the data change frequency, the server receives a target data change request and sends a target data change notification to the database, and the database automatically updates the corresponding target data; at the same time, the server distributes the target data change information to each cache node through a message queue, and each cache node automatically updates the corresponding target data after receiving the target data change information from the message queue, to ensure that the data stored in the database is consistent with the cache data in the cache node; at the same time, the server records the number of changes to the target data stored in the database; and counts the number of changes to the target data within a preset time window; and determines the data change frequency of the target data based on the number of changes to the target data.

[0038] In one implementation, the server monitors changes in target data in the database through database triggers, scheduled tasks, cloud services, etc. The technology for monitoring changes in target data is existing technology and will not be described in detail here.

[0039] In one possible implementation, when the activity frequency includes the data change frequency, the server receives a target data change request and sends a target data change notification to the database, and the database automatically updates the corresponding target data; at the same time, the server distributes the target data change information to each cache node through a message queue, and each cache node automatically updates the corresponding target data after receiving the target data change information from the message queue, to ensure that the data stored in the database is consistent with the cached data in the cache node; at the same time, the server records the timestamp of the target data change stored in the database; and counts the number of timestamps of the target data change within a preset time window; and determines the data change frequency of the target data based on the number of timestamps.

[0040] In a possible implementation, when the activity frequency includes the data access frequency, the server receives the target data access request, notifies the cache node through the message queue, and records the number of accesses to the target data cached in the cache node; and counts the number of accesses to the target data within a preset time window; and determines the data access frequency of the target data based on the number of accesses. In an implementation, the server can record the number of accesses to the target data cached in the cache node through a dictionary, a set, a named tuple, etc. This is a prior art and will not be described in detail here.

[0041] In one implementation, the server can count the number of times the target data is accessed through a scheduled task or event monitoring, which is a prior art and will not be described in detail here.

[0042] In a possible implementation, when the activity frequency includes the data access frequency, the server receives a target data access request, notifies the cache node through a message queue, and records the number of accesses and access time of the target data cached in the cache node; and counts the number of accesses to the target data from the last access time to the current access time; and determines the data access frequency of the target data based on the number of accesses. In this embodiment, the data access frequency of the target data is determined based on the last access time and the current access time, rather than based on the total number of accesses in the entire life cycle, which helps to prevent outdated data from affecting the current cache decision.

[0043] In a possible implementation, when the activity frequency includes the data access frequency, the server receives the target data access request, notifies the cache node through the message queue, and records the timestamp of the target data access cached in the cache node; and counts the number of timestamps of the target data access within the preset time window; and determines the data access frequency of the target data according to the number of timestamps. For example, the server records the timestamp and access time of the target data access cached in the cache node, and counts the number of timestamps according to the last access time and the time one hour before the last access time, and determines the data access frequency of the target data according to the number of timestamps, that is, how many times the target data has been accessed within one hour.

[0044] When the server adjusts the current cache expiration time of the target data in the cache node according to the activity frequency of the target data, in one possible implementation, the activity frequency includes the data change frequency, and the server is used to determine whether the data change frequency of the target data is greater than or equal to a first threshold. If so, the current cache expiration time of the target data is reduced; if not, the current cache expiration time of the target data is extended.

[0045] When the server adjusts the current cache expiration time of the target data in the cache node according to the activity frequency of the target data, in a possible implementation, the activity frequency includes the data access frequency, and the server is used to determine whether the data access frequency of the target data is greater than or equal to a second threshold. If it is, the current cache expiration time of the target data is extended; if it is not, the current cache expiration time of the target data is reduced.

[0046] The first threshold and the second threshold mentioned above can be set based on experience.

[0047] The above embodiments should not be regarded as limiting the protection scope of the present application, but are merely examples. The above embodiments can be freely combined to jointly adjust the current cache expiration time of the target data in the cache node.

[0048] The embodiment of the present application also provides a cache consistency management method, which is applied to a backend service, and the backend service is connected to a database and a cache node respectively. Figure 2 The flowchart shown illustrates the cache coherence management method.

[0049] Step S100: monitoring the activity frequency of target data in a database or a cache node, wherein the data stored in the database is consistent with the data cached in the cache node.

[0050] In one possible implementation, the activity frequency includes the data change frequency, and the activity frequency of the target data in the monitoring database in the above step S100 may include: the data change frequency of the target data stored in the monitoring database; or, the activity frequency includes the data access frequency, and the activity frequency of the target data in the monitoring cache node in the above step S100 may include: the data access frequency of the target data cached in the monitoring cache node.

[0051] In one embodiment, the activity frequency includes the data change frequency, and the process of monitoring the data change frequency of the target data stored in the database may include: recording the number of changes to the target data stored in the database; and counting the number of changes to the target data within a preset time window; and determining the data change frequency of the target data based on the number of changes.

[0052] In one embodiment, the activity frequency includes the data change frequency, and the process of monitoring the data change frequency of the target data stored in the database may include: recording the timestamps of the target data changes stored in the database; and counting the number of timestamps of the target data changes within a preset time window; and determining the data change frequency of the target data based on the number of timestamps.

[0053] In one possible implementation, the activity frequency includes the data access frequency, and the process of monitoring the data access frequency of the target data cached in the cache node may include: recording the number of access times of the target data cached in the cache node; and counting the number of access times of the target data within a preset time window; and determining the data access frequency of the target data based on the number of access times.

[0054] In one possible implementation, the activity frequency includes the data access frequency, and the process of monitoring the data access frequency of the target data cached in the cache node may include: recording the timestamps of the target data access cached in the cache node; and counting the number of timestamps of the target data access within a preset time window; and determining the data access frequency of the target data based on the number of timestamps.

[0055] Step S200: adjusting the current cache expiration time of the target data in the cache node according to the activity frequency of the target data.

[0056] In a possible implementation, the activity frequency includes the data change frequency. The process of adjusting the current cache expiration time of the target data in the cache node according to the activity frequency of the target data in the above step S200 may include: determining whether the data change frequency of the target data is greater than or equal to a first threshold, and if so, reducing the current cache expiration time of the target data; if not, extending the current cache expiration time of the target data.

[0057] In another possible implementation, the activity frequency includes the data access frequency. The process of adjusting the current cache expiration time of the target data in the cache node according to the activity frequency of the target data in the above step S200 may include: determining whether the data access frequency of the target data is greater than or equal to a second threshold, and if so, extending the current cache expiration time of the target data; if not, reducing the current cache expiration time of the target data.

[0058] It can be understood that the above embodiment is only one of many embodiments of the present application, and therefore, it cannot be understood as the only embodiment of the present application.

[0059] like Figure 3 As shown, Figure 3 The schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application is shown. The electronic device 20 includes: a processor 21 and a memory 22.

[0060] It should be noted that Figure 3 The components and structures of the electronic device 20 shown are merely exemplary and non-limiting. The electronic device may also have other components and structures as required.

[0061] The processor 21, the memory 22, and other components that may appear in the electronic device 20 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, the processor 21, the memory 22, and other components that may appear may be electrically connected to each other via one or more communication buses or signal lines.

[0062] The memory 22 is used to store programs, for example, a program corresponding to the cache consistency management method mentioned above.

[0063] The processor 21 is used to execute the executable modules stored in the memory 22 .

[0064] Of course, the method disclosed in any embodiment of the present application can be applied to the processor 21, or implemented by the processor 21.

[0065] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0066] In several embodiments provided in the present application, it should be understood that the disclosed method can also be implemented in other ways. The embodiments described above are merely schematic. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and a part of the module, program segment or code includes one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0067] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A cache consistency management system, It is characterized in that include: A database and a cache node, wherein the data stored in the database is consistent with the data cached in the cache node; The server is used to monitor the activity frequency of the target data in the database or the cache node, and adjust the current cache expiration time of the target data in the cache node according to the activity frequency of the target data.

2. The system according to claim 1, It is characterized in that The activity frequency includes a data change frequency, and the server is used to monitor the data change frequency of the target data stored in the database; Alternatively, the activity frequency includes a data access frequency, and the server is used to monitor the data access frequency of the target data cached in the cache node.

3. The system according to claim 2, It is characterized in that The activity frequency includes a data change frequency, and the server is used to record the number of changes of the target data stored in the database; and counting the number of changes of the target data within a preset time window; According to the number of changes, a data change frequency of the target data is determined.

4. The system according to claim 2, It is characterized in that The activity frequency includes a data change frequency, and the server is used to record a timestamp of a change in the target data stored in the database; And count the number of timestamps of target data changes within a preset time window; The data change frequency of the target data is determined according to the number of timestamps.

5. The system according to claim 2, It is characterized in that The activity frequency includes a data access frequency, and the server is used to record the number of accesses to the target data cached in the cache node; and counting the number of accesses to the target data within a preset time window; A data access frequency of the target data is determined according to the number of accesses.

6. The system according to claim 2, It is characterized in that The activity frequency includes a data access frequency, and the server is used to record a timestamp of a target data access cached in the cache node; and count the number of timestamps of the target data access within a preset time window; The data access frequency of the target data is determined according to the number of the timestamps.

7. The system according to claim 2, It is characterized in that The activity frequency includes the data change frequency. The server is used to determine whether the data change frequency of the target data is greater than or equal to a first threshold. If yes, the current cache expiration time of the target data is reduced; if no, the current cache expiration time of the target data is extended.

8. The system according to claim 2, It is characterized in that The activity frequency includes the data access frequency. The server is used to determine whether the data access frequency of the target data is greater than or equal to a second threshold. If yes, the current cache expiration time of the target data is extended; if no, the current cache expiration time of the target data is reduced.

9. A cache consistency management method, It is characterized in that Applied to backend services, the backend services are connected to the database and cache nodes respectively, including: Monitoring the activity frequency of target data in the database or cache node, wherein the data stored in the database is consistent with the data cached in the cache node; According to the activity frequency of the target data, the current cache expiration time of the target data in the cache node is adjusted.

10. An electronic device, It is characterized in that include: A memory and a processor, wherein the memory is connected to the processor; The memory is used to store programs; The processor is used to call the program stored in the memory to execute the method as claimed in claim 9.

Images