Caching system and data caching method

Abstract

The embodiment of the invention provides a caching system and a data caching method. The cache system comprises a cache controller and a server cluster. The cache controller sets a plurality of timing refresh time points, and the difference value between two adjacent timing refresh time points is within a tolerable duration. And at the regular refreshing time point, the cache controller controls all the servers in the server cluster to be refreshed at the same time, so that when the cache data is refreshed, the consistency of the cache data is improved, and the batch program processing accuracy is improved. And the difference value between two adjacent regular refreshing time points is limited, so that the freshness of the cached data can be ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a cache system and a data cache method. Background Art

[0002] As the digital transformation of banks is gradually being promoted, more and more banking businesses are gradually shifting from offline to online. At the same time, as the scale of banking business is also expanding, the amount of business data is growing rapidly. The increase in the amount of business data increases the access pressure of the database. Since each access to obtain data consumes a lot of resources, the access concurrency and response speed of the database are reduced.

[0003] At present, cache is often introduced to store a large amount of data that is read repeatedly and not frequently modified. When data is needed, it is searched from the cache. Since the cache uses a device with a faster access speed to store data, it can balance the problem of accessing data and response speed.

[0004] However, there may be scenarios where the cache is read multiple times, and the cached data can be modified, so the cached data needs to be refreshed regularly to ensure that the data in use is the result of the latest modification. However, most open source caches currently cannot implement synchronous refresh operations in distributed mode, resulting in inconsistent cache data between different servers, affecting the cache effect. Summary of the invention

[0005] The embodiments of the present application provide a cache system and a data cache method for implementing synchronous refresh operations in a distributed mode, improving the consistency of cache data between servers, and improving the cache effect.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a cache system, including: a cache controller and a server cluster;

[0008] The cache controller is in communication connection with all servers in the server cluster; each server in the server cluster includes a database and a cache area;

[0009] The cache controller is used to: determine a plurality of time points for scheduled refresh, wherein the difference between any two adjacent time points for scheduled refresh is a preset duration; and send a refresh instruction to each server in the server cluster at each time point for scheduled refresh;

[0010] Each server is used for: reading data from a corresponding database based on the refresh instruction, and caching the read data into a corresponding cache area.

[0011] Optionally, the cache area of ​​each server includes a first cache area and a second cache area;

[0012] Each server is specifically used to: determine a target cache area, and based on the refresh instruction, cache the data read from the corresponding database to the target cache area; the target cache area is one of the first cache area and the second cache area.

[0013] Optionally, each of the servers is specifically used for:

[0014] If a batch program accesses data in the first cache area in the corresponding server, determining the second cache area in the corresponding server as the target cache area;

[0015] If a batch program accesses data in the second cache area in the corresponding server, the first cache area in the corresponding server is determined as the target cache area.

[0016] Optionally, each of the servers is further used for:

[0017] If the batch program accesses the non-target cache area of ​​the corresponding server, and after all data are cached in the target cache area, a notification instruction is sent to the batch program to notify the batch program to access the target cache area.

[0018] Optionally, each of the servers further includes a decision-making device, and the decision-making device of each server is used to:

[0019] If a batch program accesses a corresponding server, an access request sent by the batch program is received; based on a preset strategy, the target cache area is determined from the first cache area and the second cache area of ​​the corresponding server.

[0020] Optionally, the decision-making device of each server is specifically used for:

[0021] Based on data freshness, access frequency and / or cache state, the target cache area is determined from the first cache area and the second cache area of ​​the corresponding server; the data freshness is related to the data loading time.

[0022] Optionally, the decision-making device of each server is further used for:

[0023] After all data are cached in the target cache area, a refresh completion instruction is received; the notification instruction is sent to the batch program to notify the batch program to access the target cache area; and non-target cache areas are cleared.

[0024] In a second aspect, an embodiment of the present application provides a data caching method, which is applied to a cache system, wherein the cache system includes: a cache controller and a server cluster; the cache controller is communicatively connected to all servers in the server cluster; each server in the server cluster includes a database and a cache area; the method includes:

[0025] The cache controller determines a plurality of time points for scheduled refresh, wherein the difference between any two adjacent time points for scheduled refresh is a preset duration; and at each time point for scheduled refresh, sends a refresh instruction to each server in the server cluster;

[0026] Each server reads data from a corresponding database based on the refresh instruction, and caches the read data in a corresponding cache area.

[0027] Optionally, the cache area of ​​each server includes a first cache area and a second cache area; and the method further includes:

[0028] Each server determines a target cache area, where the target cache area is one of the first cache area and the second cache area;

[0029] The caching the read data into the corresponding cache area includes: caching the data read from the corresponding database into the target cache area.

[0030] Optionally, each server determines a target cache area, including:

[0031] If a batch program accesses data in the first cache area in the corresponding server, determining the second cache area in the corresponding server as the target cache area;

[0032] If a batch program accesses data in the second cache area in the corresponding server, the first cache area in the corresponding server is determined as the target cache area.

[0033] In a third aspect, an embodiment of the present application provides a computer storage medium for storing a computer program, which, when executed, is used to implement the method as described in any one of the first aspects.

[0034] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to implement a method as described in any one of the first aspects.

[0035] The embodiments of the present application provide a cache system and a data cache method. The cache system includes: the cache system includes a cache controller and a server cluster. The cache controller sets multiple timed refresh time points, and the difference between two adjacent timed refresh time points is within a tolerable time length. At the timed refresh time point, the cache controller controls all servers in the server cluster to refresh at the same time, thereby improving the consistency of the cached data when refreshing the cached data and improving the accuracy of batch program processing. And by limiting the difference between two adjacent timed refresh time points, the freshness of the cached data can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 An architecture diagram of a distributed cache system provided in an embodiment of the present application;

[0038] Figure 2 An architectural diagram of a local cache system provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of the structure of a cache system provided in an embodiment of the present application;

[0040] Figure 4 A flow chart of a data caching method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In order to enable those skilled in the art to better understand the embodiments of the present application, technical terms are first introduced.

[0043] Cache: The original meaning of cache is a concept in computer hardware, which refers to a high-speed memory with faster access speed than general random access memory. At present, the concept of cache has been expanded, and cache exists between memory and hard disk, and even between hard disk and network. In short, any structure located between two hardware with large speed differences and used to coordinate the difference in data transmission speed between the two can be called cache.

[0044] Batch program: A series of programs are combined in sequence, without user interaction, and the scheduling and processing of the corresponding programs are completed regularly according to the work steps and execution order set by the batch system. It is generally used to process large amounts of data and needs to process the data step by step.

[0045] Currently, there are two data caching methods: one is distributed caching technology and the other is local caching technology.

[0046] Among them, distribution is a computing and system architecture model, which means deploying programs on multiple servers, communicating and collaborating through the network, with each server processing different data, splitting the processing of all data from one server to multiple servers processing different data to improve processing efficiency.

[0047] For example, Figure 1 An architecture diagram of a distributed cache system provided in an embodiment of the present application. The distributed system architecture includes a server cluster, a redis cache, and a backend database. The server cluster includes n servers, specifically server 1 to server n, where n is greater than or equal to 1, and n is a positive integer.

[0048] The server cluster is connected to the same redis cache. The redis cache is used to cache frequently accessed data. Specifically, when the server needs to access business data, it first reads the data from the redis cache. If the business data is read, it is directly returned. If the business data is not read, the data is read from the backend database and placed in the redis cache so that subsequent requests can be directly obtained from the cache.

[0049] When data needs to be written, all backend databases and data in the redis cache are updated, so that synchronous updates can be guaranteed, and thus the consistency of cached data can be guaranteed for all servers in the server cluster.

[0050] but Figure 1 In the cache system shown, the redis cache is separated from the application, so that the server cluster can share cache data, but it also causes additional network overhead when batch programs access the redis cache. Frequent access to data reduces user experience and application performance.

[0051] Local cache means that each server in the server cluster has its own cache area, and batch programs obtain cache data from this server. Figure 2 An architectural diagram of a local cache system provided in an embodiment of the present application. The cache system includes a database and a server cluster. Each of the n servers in the server cluster has a local cache area. Batch programs can obtain cache data from the cache area of ​​this server.

[0052] Local cache is applied inside the program and has no network overhead. However, in distributed mode, the cache data of each server cannot be shared. When the cache data needs to be refreshed at the same time, cache inconsistency problems are prone to occur, resulting in batch program processing errors.

[0053] In view of this, an embodiment of the present application provides a cache system. The cache system includes a cache controller and a server cluster. The cache controller sets multiple timed refresh time points, and the difference between two adjacent timed refresh time points is within a preset time length. At the timed refresh time point, the cache controller controls all servers in the server cluster to refresh at the same time, thereby improving the consistency of the cached data when refreshing the cached data and improving the accuracy of batch program processing. And by limiting the difference between the timed refresh time points, the freshness of the cached data can be guaranteed.

[0054] Attached Figure 3 A schematic diagram of a cache system provided in an embodiment of the present application. The cache system includes: a cache controller 101, a server cluster and a database. The server cluster includes n servers. The cache controller 101 is connected to the n servers respectively, and is used to realize quasi-real-time synchronization of cache data of the n servers.

[0055] Each server includes a decision-making device and a storage device. The storage device may include a first cache area (ie, cache area A) and a second cache area (cache area B).

[0056] When a batch program in a server accesses hotspot data, it first determines which cache area to read data from through the decision device of the server, for example, to read data from cache area A or to read data from cache area B. The decision device returns the determination result to the batch program, and the batch program accesses the hotspot data based on the return result of the decision device.

[0057] For example, when a batch program accesses hot spot data of server 1, the batch program sends an access request to the decision device of server 1. The decision device of server 1 determines to read the cache area and returns a determination result. If the determination result returned is to read the data in cache area A, the batch program reads the data from cache area A of server 1.

[0058] It should be noted that at a certain moment, the batch program will only read data from one cache area of ​​a server, and the other cache area will be temporarily unavailable. For example, the decision device returns cache area A to the batch program, and the batch program reads data from cache area A of the server, and cache area B is temporarily unavailable.

[0059] Among them, the decision-making device of the server can determine which cache area to read data from based on preset strategies, such as the freshness of the data (that is, the closer the data loading time is to the current time, the higher the data freshness), access frequency and the status of the cache area (for example, whether the cache area is loading data).

[0060] In the embodiment of the present application, the cache controller 101 presets a tolerable time length. That is, within the tolerable time length, the data loaded by the server are all the latest data. For example, the tolerable time length is 5 minutes, that is, the data loaded by the server within 5 minutes are all the latest data.

[0061] The cache controller 101 sets multiple scheduled refresh time points based on the tolerable duration. At the scheduled refresh time point, the cache controller 101 sends a refresh instruction to each server in the server cluster to ensure that all servers in the server cluster refresh the cache at the same time point, ensuring that the cached data on all servers are consistent and the latest data.

[0062] Specifically, the time interval between any two adjacent time refresh time points among the multiple time refresh time points is the tolerable time length. For example, the cache controller 101 sets three adjacent time refresh time points, T1, T2 and T3, and the tolerable time length is 5 minutes, then T2-T1=5 minutes, T3-T2=5 minutes.

[0063] In the embodiment of the present application, by limiting the time interval between any two adjacent scheduled refresh time points to a tolerable duration, it can be ensured that the data loaded by each server is the latest data.

[0064] In the embodiment of the present application, after the cache controller 101 issues a refresh instruction to each server in the server cluster, each server performs the following operations:

[0065] Step 1: The server reads data from the database and stores the read data in the target cache.

[0066] In one example, the target cache area may be a cache area specified by the server, for example, cache area A may be specified as the target cache area, or cache area B may be specified as the target cache area. Alternatively, cache area A may be specified as the target cache area at a first preset time, and cache area B may be specified as the target cache area at a second preset time.

[0067] In one example, to avoid affecting the currently executed access business when changing cache data, when the server determines that the currently accessed cache area is cache area A, the server determines cache area B as the target cache area. Similarly, when the cache area accessed by the batch program is cache area B, the server determines cache area A as the target cache area.

[0068] In the embodiment of the present application, if cache area A is the target cache area, cache area B is the non-target cache area; if cache area B is the target cache area, cache area A is the non-target cache area.

[0069] Step 2: When all hot data that need to be cached in the server are loaded into the target cache area, a refresh completion instruction is sent to the server's decision device, and the server's decision device sends a notification instruction to the batch program to complete the refresh of all data through the server's target cache area B.

[0070] It can be understood that the data in cache area B is the latest data at this time.

[0071] Step 3: The batch program reads data from the target buffer.

[0072] At this time, the batch program switches from reading data from the non-target buffer area to reading data from the target buffer area.

[0073] Furthermore, since the data in the non-target cache area is not the latest data, the data in the non-target cache area may be cleared to release cache resources.

[0074] In summary, an embodiment of the present application provides a cache system. The cache system includes a cache controller and a server cluster. The cache controller sets multiple timed refresh time points, and the difference between two adjacent timed refresh time points is within a tolerable time length. At the timed refresh time point, the cache controller controls all servers in the server cluster to refresh at the same time, thereby improving the consistency of cache data when refreshing cache data and improving the accuracy of batch program processing.

[0075] In addition, the present application also provides a data caching method, which is applied to the attached Figure 3 In the cache system shown.

[0076] Attached Figure 4 A flow chart of a data caching method provided in an embodiment of the present application, the method includes the following contents:

[0077] S410 , the cache controller 101 determines a plurality of scheduled refresh time points.

[0078] The cache controller determines a plurality of time points for scheduled refresh, wherein the difference between any two adjacent time points for scheduled refresh among the plurality of time points for scheduled refresh is a preset duration; wherein the preset duration is a tolerable duration.

[0079] S420: At the scheduled refresh time point, a refresh instruction is sent to all servers in the server cluster.

[0080] The cache controller 101 sends a refresh instruction to all servers at the scheduled refresh time point.

[0081] S430: Each server in the server cluster reads data from the corresponding database, and caches the read data in a cache area.

[0082] The cache area may be a preset buffer space in the server.

[0083] Furthermore, the cache area of ​​the server includes a first cache area and a second cache area. The server stores the read data in the target cache area.

[0084] In one example, the target cache area may be a cache area specified by the server, for example, specifying the first cache area as the target cache area, or specifying the second cache area as the target cache area, or specifying that at a first preset time, the first cache area is used as the target cache area, and at a second preset time, the second cache area is used as the target cache area.

[0085] In one example, in order to avoid affecting the currently executing access business when changing cache data, if a batch program accesses data in the first cache area in the corresponding server, the second cache area in the corresponding server is determined as the target cache area; if a batch program accesses data in the second cache area in the corresponding server, the first cache area in the corresponding server is determined as the target cache area.

[0086] Furthermore, after the hot spot data of each server is loaded into the target cache area, a refresh completion instruction is sent to the decision device of the server.

[0087] In addition, if the batch program accesses the non-target cache area of ​​the corresponding server, and after all data is cached in the target cache area, a notification instruction is sent to the batch program to notify the batch program to access the target cache area.

[0088] Optionally, each server further includes a decision device. If a batch program accesses a corresponding server, the decision device receives an access request sent by the batch program; based on a preset strategy, the target cache area is determined from the first cache area and the second cache area of ​​the corresponding server.

[0089] In addition, the determining the target cache area from the first cache area and the second cache area of ​​the corresponding server based on a preset strategy includes:

[0090] Based on data freshness, access frequency and / or cache state, the target cache area is determined from the first cache area and the second cache area of ​​the corresponding server; the data freshness is related to the data loading time.

[0091] Optionally, after all data are cached in the target cache area, sending a notification instruction to the batch program to notify the batch program to access the target cache area includes:

[0092] After all data are cached in the target cache area, a refresh completion instruction is received; the notification instruction is sent to the batch program to notify the batch program to access the target cache area; and non-target cache areas are cleared.

[0093] The embodiment of the present application provides a caching method, in which a cache controller is used to set multiple timed refresh time points, and the difference between two adjacent timed refresh time points is within a tolerable duration. At the timed refresh time point, the cache controller controls all servers in the server cluster to refresh simultaneously, thereby improving the consistency of cached data and the accuracy of batch program processing when refreshing cached data. And by limiting the difference between two adjacent timed refresh time points, the freshness of the cached data can be guaranteed.

[0094] For the convenience of description, the above system is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0095] It is understandable that in the specific implementation of the present application, related data such as user information and historical user information are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0096] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0097] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0098] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0099] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0100] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cache system, characterized in that: include: Cache controller, server cluster and database; The cache controller is connected to all servers in the server cluster; Each server in the server cluster includes a cache area; The cache controller is used to: determine a plurality of time points for scheduled refresh, wherein the time interval between any two adjacent time points for scheduled refresh is a preset time length; and send a refresh instruction to each server in the server cluster at each time point for scheduled refresh; Each server is used for: reading data from the database based on the refresh instruction, and caching the read data into a corresponding cache area.

2. The system according to claim 1, characterized in that The cache area of ​​each server includes a first cache area and a second cache area; Each server is specifically used to: determine a target cache area, and based on the refresh instruction, cache the data read from the corresponding database to the target cache area; the target cache area is one of the first cache area and the second cache area.

3. The system according to claim 2, characterized in that Each of the servers is specifically used for: If a batch program accesses data in the first cache area in the corresponding server, determining the second cache area in the corresponding server as the target cache area; If a batch program accesses data in the second cache area in the corresponding server, the first cache area in the corresponding server is determined as the target cache area.

4. The system according to claim 2, characterized in that Each of the servers is also used to: If the batch program accesses the non-target cache area of ​​the corresponding server, and after all data are cached in the target cache area, a notification instruction is sent to the batch program to notify the batch program to access the target cache area.

5. The system according to any one of claims 2 to 4, characterized in that: Each of the servers further comprises a decision-making device, wherein the decision-making device of each server is used to: If a batch program accesses a corresponding server, an access request sent by the batch program is received; based on a preset strategy, the target cache area is determined from the first cache area and the second cache area of ​​the corresponding server.

6. The system according to claim 5, characterized in that The decision-making device of each server is specifically used for: Based on data freshness, access frequency and / or cache state, the target cache area is determined from the first cache area and the second cache area of ​​the corresponding server; the data freshness is related to the data loading time.

7. The system according to claim 4, characterized in that The decision-making device of each server is also used for: After all data are cached in the target cache area, a refresh completion instruction is received; the notification instruction is sent to the batch program to notify the batch program to access the target cache area; and non-target cache areas are cleared.

8. A data caching method, characterized in that: Applied to a cache system, the cache system comprises: a cache controller, a server cluster and a database; the cache controller is in communication connection with all servers in the server cluster; each server in the server cluster comprises a cache area; the method comprises: The cache controller determines a plurality of time points for scheduled refresh, wherein the difference between any two adjacent time points for scheduled refresh is a preset duration; and at each time point for scheduled refresh, sends a refresh instruction to each server in the server cluster; Each of the servers reads data from the database based on the refresh instruction, and caches the read data in a corresponding cache area.

9. The method according to claim 8, characterized in that The cache area of ​​each server includes a first cache area and a second cache area; the method further includes: Each server determines a target cache area, where the target cache area is one of the first cache area and the second cache area; The caching the read data into the corresponding cache area includes: caching the data read from the corresponding database into the target cache area.

10. The method according to claim 9, characterized in that Each server determines a target cache area, including: If a batch program accesses data in the first cache area in the corresponding server, determining the second cache area in the corresponding server as the target cache area; If a batch program accesses data in the second cache area in the corresponding server, the first cache area in the corresponding server is determined as the target cache area.

Images