A service process processing system and method using multi-level data cache

Abstract

The present invention proposes a service process processing system and method using multi-level data caching, relating to the fields of cloud computing and distributed storage technology. The system comprises: a local cache unit, a Redis centralized cache unit, a database and a service process processing module; wherein the local cache unit, the Redis centralized cache unit and the database adopt a multi-level data caching mode to store data, and the access priority from high to low is: local cache unit, Redis centralized cache unit and database; the service process processing module is used to preferentially access the local cache unit when the service process is started; if no data is hit in the local cache unit, the Redis centralized cache unit is accessed; if the data is hit, the data is read and transaction processing is continued; if no data is hit in the Redis centralized cache unit, the database is accessed; if the data is hit, the data is read and transaction processing is continued.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing and distributed storage technology, and in particular to a service process processing system and method utilizing multi-level data cache. Background Art

[0002] Currently, high availability in distributed caches is essentially achieved through data redundancy. Using multiple copies of stored data distributes database read and write pressure, preventing cache failure or avalanche. In existing technologies, client-side solutions are often used to achieve high availability in distributed caches. This approach configures multiple cache nodes on the client side and implements distributed cache write and read algorithms, thereby improving cache availability. When writing data, the client distributes the data to be written to the cache across multiple nodes, a process known as data sharding. When reading data, Memcached uses a master-slave approach to achieve fault tolerance. In the event of a master node failure, slave nodes provide a backup to prevent cache failure.

[0003] While the client-side data sharding and writing strategy can alleviate the storage and access pressure on cache nodes, it also complicates cache usage. Typically, data sharding uses a consistent hashing algorithm to scatter data across different storage nodes. Excessive number of nodes increases the likelihood of problems. Furthermore, the consistent hashing algorithm can cause service processes to read dirty data. For example, if cache A's data is updated and cannot connect to the client due to a failure, the client will write the updated data to cache B. When cache A restores its connection to the client, the client will read the dirty data from cache A. Memcached, used in the client-side solution, is an in-memory database, but it does not support master-slave mode. Cache distribution strategies must be written on the client side, and data synchronization is not supported. Therefore, if a single machine fails in a production environment, some services may be affected.

[0004] In summary, there is an urgent need for a technical solution that can overcome the above-mentioned defects, improve the high availability of distributed cache, and ensure the stability of business processing system operation. Summary of the Invention

[0005] To address the challenges of existing technologies, this invention proposes a service process processing system and method utilizing multi-level data caching. Built on a three-level storage model consisting of "local cache - Redis centralized cache - database," this invention leverages the high availability of Redis and the database. Even if the Redis cluster or database becomes unavailable, service processes can still access parameter data. This ensures that most transactions can continue to operate smoothly for a period of time, maintaining system stability and buying time for fault repairs.

[0006] In a first aspect of an embodiment of the present invention, a service process processing system using multi-level data cache is proposed, the system comprising: a local cache unit, a Redis centralized cache unit, a database and a service process processing module; wherein,

[0007] The local cache unit, the Redis centralized cache unit and the database use a multi-level data cache mode to store data, and the access priority from high to low is: local cache unit, Redis centralized cache unit and database;

[0008] The service process processing module is used to give priority to accessing the local cache unit when the service process is started;

[0009] If the data cannot be found in the local cache unit, the Redis centralized cache unit is accessed; if the data is found, the data is read and the transaction processing continues;

[0010] If the data cannot be hit in the Redis centralized cache unit, the database is accessed; if the data is hit, the data is read and the transaction processing continues.

[0011] Furthermore, the local cache unit is a first-level cache, which is used to cache data with an access frequency greater than a set value.

[0012] Furthermore, the Redis centralized cache unit is a secondary cache, which is used to cache a certain amount of data in the database.

[0013] Furthermore, the system also includes:

[0014] The data synchronization module is used for regular multi-level data caching. When the data changes, parameter synchronization technology is used to keep the data in the local cache, Redis centralized cache unit and database consistent.

[0015] Furthermore, the Redis centralized cache unit adopts a cluster mode, and the cluster mode uses hash slots. When the number of hash slots of storage nodes in the Redis centralized cache unit is deleted or changed, the Redis centralized cache unit remains available.

[0016] Furthermore, in the cluster mode, the Redis centralized cache unit includes multiple Redis master nodes, each Redis master node is correspondingly provided with at least one Redis slave node, and when the Redis master node is inaccessible, access is performed through the Redis slave node.

[0017] Furthermore, the database adopts a master-slave multi-copy mode, is equipped with a database master node and at least one database slave node, and data is synchronized in real time through Binlog; when the database master node is unavailable, it automatically switches to the database slave node.

[0018] Furthermore, the service process processing module is also used to:

[0019] When the Redis centralized cache unit is unavailable and the service process cannot hit the data in the local cache unit, the service process directly reads the data from the database.

[0020] Furthermore, the service process processing module is also used to:

[0021] When a database failure occurs, the local cache and Redis centralized cache unit support service process access and keep transactions processing.

[0022] Furthermore, the service process processing module is also used to:

[0023] When both the database and the Redis centralized cache unit are inaccessible, the local cache supports service process access and keeps the transaction processing going.

[0024] Furthermore, the Redis centralized cache unit includes a preset Redis cluster proxy component and multiple data partitions, each data partition includes multiple Redis master nodes and Redis slave nodes; wherein,

[0025] When a service process initiates an access request to the Redis centralized cache unit, the target data partition that is not down is determined through the Redis cluster proxy component and the data key value in the access request; the access request is routed to the target data partition, and the data key value is processed to obtain a hash slot number; the data access request is routed to the Redis master node corresponding to the hash slot number in the target data partition. If the Redis master node is inaccessible, access is performed through the Redis slave node.

[0026] Furthermore, a master-slave synchronization unit is provided in the database; wherein,

[0027] The master-slave synchronization unit is used to monitor whether a new Binlog file is generated by the database master node; if a new Binlog file is generated, the newly generated Binlog file is packaged and compressed and sent to the database standby node; if no new Binlog file is generated, the monitoring continues; the database standby node unpacks and verifies the received compressed Binlog file, obtains complete data, and saves it to the storage area.

[0028] In a second aspect of an embodiment of the present invention, a service process processing method using a multi-level data cache is proposed, the method comprising:

[0029] Set up a multi-level data cache mode. In this mode, the access priority is from high to low: local cache unit, Redis centralized cache unit, and database.

[0030] When the service process starts, it gives priority to accessing the local cache unit;

[0031] If the data cannot be found in the local cache unit, the Redis centralized cache unit is accessed; if the data is found, the data is read and the transaction processing continues;

[0032] If the data cannot be hit in the Redis centralized cache unit, the database is accessed; if the data is hit, the data is read and the transaction processing continues.

[0033] Furthermore, the method further comprises:

[0034] Regular multi-level data caching. When data changes, parameter synchronization technology is used to keep the data in the local cache, Redis centralized cache unit and database consistent.

[0035] Furthermore, the Redis centralized cache unit adopts a cluster mode, and the cluster mode uses hash slots. When the number of hash slots of storage nodes in the Redis centralized cache unit is deleted or changed, the Redis centralized cache unit remains available.

[0036] Furthermore, in the cluster mode, the Redis centralized cache unit includes multiple Redis master nodes, each Redis master node is correspondingly provided with at least one Redis slave node, and when the Redis master node is inaccessible, access is performed through the Redis slave node.

[0037] Furthermore, the database adopts a master-slave multi-copy mode, is equipped with a database master node and at least one database slave node, and data is synchronized in real time through Binlog; when the database master node is unavailable, it automatically switches to the database slave node.

[0038] Furthermore, the method further comprises:

[0039] When the Redis centralized cache unit is unavailable and the service process cannot hit the data in the local cache unit, the service process directly reads the data from the database.

[0040] Furthermore, the method further comprises:

[0041] When a database failure occurs, the local cache and Redis centralized cache unit support service process access and keep transactions processing.

[0042] Furthermore, the method further comprises:

[0043] When both the database and the Redis centralized cache unit are inaccessible, the local cache supports service process access and keeps the transaction processing going.

[0044] Furthermore, the Redis centralized cache unit includes a preset Redis cluster proxy component and multiple data partitions, each data partition includes multiple Redis master nodes and Redis slave nodes; wherein,

[0045] When a service process initiates an access request to the Redis centralized cache unit, the target data partition that is not down is determined through the Redis cluster proxy component and the data key value in the access request; the access request is routed to the target data partition, and the data key value is processed to obtain a hash slot number; the data access request is routed to the Redis master node corresponding to the hash slot number in the target data partition. If the Redis master node is inaccessible, access is performed through the Redis slave node.

[0046] Furthermore, a master-slave synchronization unit is provided in the database; wherein,

[0047] The master-slave synchronization unit is used to monitor whether a new Binlog file is generated by the database master node; if a new Binlog file is generated, the newly generated Binlog file is packaged and compressed and sent to the database standby node; if no new Binlog file is generated, the monitoring continues; the database standby node unpacks and verifies the received compressed Binlog file, obtains complete data, and saves it to the storage area.

[0048] In a third aspect of an embodiment of the present invention, a computer device is proposed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a service process processing method utilizing a multi-level data cache when executing the computer program.

[0049] In a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a service process processing method using a multi-level data cache is implemented.

[0050] The service process processing system and method using multi-level data caching proposed by the present invention can ensure that the service process can read parameter data by setting a multi-level storage scheme of local cache, Redis centralized cache and database, reduce the impact of failures in the production environment on some businesses, and maintain stable operation of the system; the local cache is used as a first-level cache to ensure that the service process can read most of the required parameters from it, avoid frequent reading of the second-level cache or database, and when the Redis centralized cache or database fails to serve, it ensures that most transactions can persist for a period of time without being affected by the failure and proceed smoothly, so that transactions can continue without direct failure, thus achieving high availability; Redis cache is used as a first-level cache to ensure that the service process can read most of the required parameters from it, avoid frequent reading of the second-level cache or database, and when the Redis centralized cache or database fails to serve, it ensures that most transactions can persist for a period of time without being affected by the failure and proceed smoothly, so that transactions can continue without direct failure, thus achieving high availability; It is a secondary cache. If the service process cannot hit data in the local cache unit, it can access the Redis centralized cache unit, and when it is unavailable, the service process can directly read the database; both the Redis cluster and the database have high availability. The Redis cluster mode of the Redis centralized cache adopts the hash slot algorithm and the master-slave replication model to ensure that changes and downtime of the storage node will not affect the external services of the entire cluster; the master-slave multi-copy mode of the database can switch to the database slave node when the database master node is unavailable, which is transparent to the application and enables the application to read and write data from the database without being affected; the overall solution can maintain the stable operation of the system and buy time for fault repair. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 1 is a schematic diagram of a service process processing system architecture using multi-level data cache according to an embodiment of the present invention.

[0053] Figure 2 It is a schematic diagram of the architecture of a service process processing system using multi-level data cache according to a specific embodiment of the present invention.

[0054] Figure 3 It is a schematic diagram of the architecture of Redis centralized cache according to a specific embodiment of the present invention.

[0055] Figure 4 FIG. 4 is a schematic diagram of the database architecture of a specific embodiment of the present invention.

[0056] Figure 5 The figure is a flow chart of a service process processing method using multi-level data cache according to an embodiment of the present invention.

[0057] Figure 6The figure is a flow chart of a service process processing method using multi-level data cache according to a specific embodiment of the present invention.

[0058] Figure 7 It is a flowchart of a service process processing method using multi-level data cache according to another specific embodiment of the present invention.

[0059] Figure 8 It is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0061] Those skilled in the art will appreciate that the embodiments of the present invention may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0062] According to an embodiment of the present invention, a service process processing system and method using multi-level data cache are proposed, which relate to the fields of cloud computing and distributed storage technology.

[0063] In banking business scenarios, there is a type of parameters that change less frequently but are read frequently by various application components, such as currency type, exchange rate, institution address, etc. In order to enable such parameters to be read quickly and correctly, and to avoid negative impacts on transactions due to failure to read or reading incorrect data, the present invention proposes a service process processing solution that utilizes multi-level data caching. In response to the demand for high availability of data reading, the present invention establishes a three-level storage mode of "local cache-Redis centralized cache-database". The present invention fully utilizes the high availability of Redis and databases. When the Redis cluster or database is unavailable, the service process can still read the parameter data, ensuring that most transactions can proceed smoothly for a period of time without being affected by failures, maintaining stable system operation, and buying time for fault repair.

[0064] In the embodiments of the present invention, the following terms need to be explained:

[0065] Memcached: An in-memory database with high read and write performance, a single data type, support for client-based distributed clustering, and consistent hashing. Its multi-core architecture and multi-threaded read and write performance are high. However, its drawbacks include a lack of persistence, potential cache penetration due to node failures, client-side implementation of distributed data, difficulty synchronizing data across data centers, and complex architecture expansion.

[0066] Redis: A remote dictionary service, it is an open-source, network-friendly, in-memory, and persistent log-based key-value database written in ANSI C. It offers APIs in multiple languages. It supports more value types than Memcached, including strings, lists, sets, zsets, and hashes. It offers high-performance read and write capabilities, support for multiple data types, data persistence, a highly available architecture, customizable virtual memory, distributed sharding clusters, and exceptional single-threaded read and write performance.

[0067] High availability: Improve system and application availability by minimizing downtime caused by routine maintenance operations and sudden system crashes.

[0068] CAP principle: In a distributed system, at most two of the three elements of consistency, availability, and partition tolerance can be achieved at the same time, and it is impossible to take all three into account.

[0069] The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments of the present invention.

[0070] Figure 1 FIG. 1 is a schematic diagram of a service process processing system architecture using multi-level data cache according to an embodiment of the present invention. Figure 1 As shown, the system includes: a local cache unit 110, a Redis centralized cache unit 120, a database 130 and a service process processing module 140; wherein,

[0071] The local cache unit 110, the Redis centralized cache unit 120 and the database 130 use a multi-level data cache mode to store data, and the access priority from high to low is: local cache unit 110, Redis centralized cache unit 120 and database 130;

[0072] The service process processing module 140 is configured to preferentially access the local cache unit 110 when the service process is started;

[0073] If the data cannot be hit in the local cache unit 110, the Redis centralized cache unit 120 is accessed; if the data is hit, the data is read and the transaction processing continues;

[0074] If the data cannot be hit in the Redis centralized cache unit 120, the database 130 is accessed; if the data is hit, the data is read and the transaction processing continues.

[0075] The multi-level data caching architecture proposed in this paper embodies a high-availability design philosophy. The three levels of storage (local cache, Redis centralized cache, and database) complement each other at different storage levels. This ensures that service processes can continue to complete transactions even if one or two storage levels become unavailable, significantly improving the robustness and robustness of the system. Locally, both the Redis cluster model and the database itself offer high-availability solutions that can prevent their own unavailability.

[0076] In order to explain the service process processing system using multi-level data cache more clearly, the following describes each part in detail.

[0077] In this embodiment, the local cache unit is a first-level cache, which is used to cache data with an access frequency greater than a set value; the local cache unit can ensure that the service process can read most of the required parameters from it, avoiding frequent reading of the second-level cache (Redis centralized cache) or database.

[0078] In this embodiment, the Redis centralized cache unit is a secondary cache, which is used to cache a certain amount of data in the database.

[0079] When the Redis centralized cache unit is unavailable, the service process can read the database directly. However, the transaction performance and concurrency may be reduced, but it still remains available.

[0080] When the database is in an extreme state where it cannot read or write, the local cache and Redis centralized cache can satisfy most read parameter requests, but dirty data may be read. Considering the CAP theory, the best availability and partition tolerance can be achieved, which is unavoidable.

[0081] A more extreme case is that both the Redis cluster and the database are unavailable. In this case, the local cache can support read data requests for a period of time, avoiding immediate system downtime and buying time for troubleshooting and repair.

[0082] Based on the above situation, the service process processing module 140 is further configured to:

[0083] When the Redis centralized cache unit is unavailable and the service process cannot hit the data in the local cache unit, the service process directly reads the data from the database.

[0084] When a database failure occurs, the local cache and Redis centralized cache unit support service process access and keep transactions processing.

[0085] When both the database and the Redis centralized cache unit are inaccessible, the local cache supports service process access and keeps the transaction processing going.

[0086] In this embodiment, the Redis centralized cache unit adopts a cluster mode, and the cluster mode uses hash slots. When the number of hash slots of a storage node in the Redis centralized cache unit is deleted or changed, the Redis centralized cache unit remains available.

[0087] In the cluster mode, the Redis centralized cache unit includes multiple Redis master nodes, each Redis master node is correspondingly provided with at least one Redis slave node, and when the Redis master node is inaccessible, access is performed through the Redis slave node.

[0088] Specifically, the Redis centralized cache unit includes a preset Redis cluster proxy component and multiple data partitions, each data partition includes multiple Redis master nodes and Redis slave nodes; wherein,

[0089] When a service process initiates an access request to the Redis centralized cache unit, the target data partition that is not down is determined through the Redis cluster proxy component and the data key value in the access request; the access request is routed to the target data partition, and the data key value is processed to obtain a hash slot number; the data access request is routed to the Redis master node corresponding to the hash slot number in the target data partition. If the Redis master node is inaccessible, access is performed through the Redis slave node.

[0090] In this embodiment, the database adopts a master-slave multi-copy mode, is equipped with a database master node and at least one database slave node, and data is synchronized in real time through Binlog; when the database master node is unavailable, it automatically switches to the database slave node.

[0091] Specifically, the database is provided with a master-slave synchronization unit; wherein,

[0092] The master-slave synchronization unit is used to monitor whether a new Binlog file is generated by the database master node; if a new Binlog file is generated, the newly generated Binlog file is packaged and compressed and sent to the database standby node; if no new Binlog file is generated, the monitoring continues; the database standby node unpacks and verifies the received compressed Binlog file, obtains complete data, and saves it to the storage area.

[0093] The present invention proposes a service process processing system that utilizes multi-level data caching. When a service process starts, it can directly read parameter data from the local cache, which stores the most popular parameter data. If the local cache encounters a problem and becomes unreadable or writable, the service process typically terminates. To improve the efficiency of reading cached data, the present invention provides a secondary cache, the Redis centralized cache, between the local cache and the database. This allows the local cache to read the Redis centralized cache first when no parameter data is being read, thus avoiding cache penetration that would occur if the database were read directly.

[0094] refer to Figure 2 , is a schematic diagram of a service process processing system architecture using multi-level data cache according to a specific embodiment of the present invention. Figure 2 As shown, the system also includes:

[0095] The data synchronization module is used for regular multi-level data caching. When the data changes, parameter synchronization technology is used to keep the data in the local cache, Redis centralized cache unit and database consistent.

[0096] The three-level storage solution adopted by the present invention achieves high availability in architecture, and the distributed system parameter synchronization solution implemented by this unit ensures the consistency of data in the local cache, Redis cache and database, avoiding the service process from reading dirty data.

[0097] From an architectural perspective, the local cache serves as the data read by the service process. If it is unavailable, the service process cannot proceed. Redis Cluster itself supports a master-slave replication mechanism, which enables read-write separation and disaster recovery, demonstrating high availability. The database's high availability is reflected in its master-slave multi-replica model, which ensures that if the database master node fails, it can be promptly switched to the backup node, preventing a complete database outage. Distributed high availability is reflected from local modules to the overall architecture.

[0098] It should be noted that although the detailed description above mentions several modules of the service process processing system utilizing multi-level data caching, this division is merely exemplary and not mandatory. In practice, according to embodiments of the present invention, the features and functions of two or more modules described above may be embodied in a single module. Conversely, the features and functions of a single module described above may be further divided and embodied by multiple modules.

[0099] Compared with existing client solutions, the multi-level cache solution proposed in this invention focuses on improving the high availability of distributed cache on the server side, fully utilizing the high availability of Redis cluster and the database itself, realizing mutual support in availability of different storage levels, and improving the availability of system cache as a whole.

[0100] The cache of the present invention includes three levels: local cache, Redis centralized cache and database. The service process will first access the local cache when reading parameters. If the data cannot be hit, it needs to read the Redis cache upward. If the data still cannot be hit, it will access the database.

[0101] From the overall architecture perspective, when a database failure occurs, the local cache and Redis cache can support service process access to ensure that transactions continue;

[0102] When the Redis cache is unavailable, the service process can directly access the database to read data;

[0103] Even if both the Redis cache and the database are inaccessible, the local cache can still support the access parameter requirements of most transactions because it stores the hottest parameter data.

[0104] refer to Figure 3 , is a schematic diagram of the architecture of Redis centralized cache according to a specific embodiment of the present invention.

[0105] like Figure 3 As shown in the figure, the high availability of Redis cache is reflected in the cluster mode. The cluster mode uses hash slots instead of the consistent hashing algorithm. This ensures that deleting or changing the number of hash slots of a storage node will not cause cluster unavailability, and also makes it easier to delete or add storage nodes.

[0106] Furthermore, the master-slave replication model in cluster mode ensures that the cluster remains available even when some nodes fail or most nodes are unable to communicate. For example, if a cluster has three master nodes, A, B, and C, each with one slave node, A1, B1, and C1, then if a node, such as B, fails, the cluster will elect B1 as the new master node to continue service. This ensures that the entire cluster remains available even if it cannot find a slot in B. In summary, the design and implementation of Redis cluster mode ensure its high availability.

[0107] refer to Figure 4 , which is a schematic diagram of the database architecture of a specific embodiment of the present invention.

[0108] like Figure 4 As shown, the database in the multi-level cache solution also has high availability. The database can adopt a master-slave multi-copy mode, setting a master node and at least one slave node (backup 1, backup 2). When the master node is unavailable, it can automatically switch to the slave node to ensure that the service process can read data from the database when it cannot read data from the local buffer and the Redis centralized cache, avoiding the service process from terminating due to failure to read data.

[0109] The data of the master node and the slave node are synchronized in real time through Binlog.

[0110] During Binlog real-time synchronization, the master node must enable binary logging to record any events that modify database data.

[0111] The slave node starts a thread (I / OThread) and acts as a MySQL client, requesting events in the binary log file (Binary log) of the master node through the MySQL protocol.

[0112] The master node starts a thread (dump thread) to check the events in its own binary log and compare them with the position requested by the other party. If there is no request position parameter, the master node will send the first event in the first log file to the slave node one by one.

[0113] The slave node receives data from the master node and places it in the relay log file. It also records the location of the binary log file within which the request was sent to the master node (the master node may have multiple binary log files, which will be explained in detail later).

[0114] Start another thread (SQL Thread) from the node, read the events in the Relay log, and execute them again locally.

[0115] In summary, the Redis cluster mode uses a hash slot algorithm and a master-slave replication model to ensure that changes and downtime of storage nodes will not affect the external services of the entire cluster.

[0116] The master-slave multi-copy mode of the database ensures that when the master node is unavailable, it can switch to the slave node. This is transparent to the application, and the application can still read and write data from the database without being affected.

[0117] Both the Redis cluster and the database are highly available.

[0118] After introducing the system of the exemplary embodiment of the present invention, next, reference is made to Figure 5 A service process processing method using multi-level data cache according to an exemplary embodiment of the present invention is introduced.

[0119] The implementation of the service process processing method using multi-level data cache can refer to the implementation of the above system, and the repeated parts will not be repeated.

[0120] Based on the same inventive concept, the present invention also proposes a service process processing method using multi-level data cache, such as Figure 5 As shown, the method includes:

[0121] S501, setting a multi-level data cache mode, wherein, in the multi-level data cache mode, the access priority from high to low is: local cache unit, Redis centralized cache unit and database;

[0122] S502, when the service process starts, the local cache unit is accessed first;

[0123] S503, if the data cannot be found in the local cache unit, access the Redis centralized cache unit; if the data is found, read the data and continue transaction processing;

[0124] S504, if the data cannot be hit in the Redis centralized cache unit, access the database; if the data is hit, read the data and continue transaction processing.

[0125] In this embodiment, reference Figure 6 , which is a flow chart of a service process processing method using multi-level data cache according to a specific embodiment of the present invention.

[0126] like Figure 6 As shown, the method further includes:

[0127] S601, regular multi-level data caching, when the data changes, use parameter synchronization technology to keep the data in the local cache, Redis centralized cache unit and database consistent.

[0128] In this embodiment, the Redis centralized cache unit adopts a cluster mode, and the cluster mode uses hash slots. When the number of hash slots of a storage node in the Redis centralized cache unit is deleted or changed, the Redis centralized cache unit remains available.

[0129] In this embodiment, in the cluster mode, the Redis centralized cache unit includes multiple Redis master nodes, each Redis master node is correspondingly provided with at least one Redis slave node, and when the Redis master node is inaccessible, access is performed through the Redis slave node.

[0130] In this embodiment, the database adopts a master-slave multi-copy mode, is equipped with a database master node and at least one database slave node, and data is synchronized in real time through Binlog; when the database master node is unavailable, it automatically switches to the database slave node.

[0131] Specifically, the Redis centralized cache unit includes a preset Redis cluster proxy component and multiple data partitions, each data partition includes multiple Redis master nodes and Redis slave nodes; wherein,

[0132] When a service process initiates an access request to the Redis centralized cache unit, the target data partition that is not down is determined through the Redis cluster proxy component and the data key value in the access request; the access request is routed to the target data partition, and the data key value is processed to obtain a hash slot number; the data access request is routed to the Redis master node corresponding to the hash slot number in the target data partition. If the Redis master node is inaccessible, access is performed through the Redis slave node.

[0133] In this embodiment, the database is provided with a master-slave synchronization unit; wherein,

[0134] The master-slave synchronization unit is used to monitor whether a new Binlog file is generated by the database master node; if a new Binlog file is generated, the newly generated Binlog file is packaged and compressed and sent to the database standby node; if no new Binlog file is generated, the monitoring continues; the database standby node unpacks and verifies the received compressed Binlog file, obtains complete data, and saves it to the storage area.

[0135] In this embodiment, reference Figure 7 , is a flow chart of a service process processing method using multi-level data cache according to another specific embodiment of the present invention.

[0136] like Figure 7 As shown, the method further includes:

[0137] S701: When the Redis centralized cache unit is unavailable and the service process cannot hit data in the local cache unit, the service process directly reads data from the database.

[0138] S702, when a database failure occurs, the local cache and the Redis centralized cache unit support service process access to keep the transaction processing going.

[0139] S703, when both the database and the Redis centralized cache unit are inaccessible, the local cache supports service process access and keeps the transaction processing going.

[0140] In summary, during the service process, the service process will first access the local cache to read parameters. If the data cannot be hit, it needs to read the Redis cache upwards. If the data still cannot be hit, it will access the database.

[0141] Based on a multi-level data cache architecture, when a database failure occurs, the local cache and Redis cache can support service process access to ensure that transactions continue; when the Redis cache is unavailable, the service process can directly access the database to read data; even if both the Redis cache and the database are inaccessible, the local cache can still support the access parameter requirements of most transactions because it stores the most popular parameter data.

[0142] It should be noted that although the operations of the method of the present invention are described in a specific order in the above embodiments and drawings, this does not require or imply that these operations must be performed in this specific order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0143] Based on the above invention concept, Figure 8 As shown, the present invention also proposes a computer device 800, including a memory 810, a processor 820 and a computer program 830 stored in the memory 810 and executable on the processor 820, wherein the processor 820 implements the aforementioned service process processing method utilizing multi-level data cache when executing the computer program 830.

[0144] Based on the aforementioned inventive concept, the present invention proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the aforementioned service process processing method using multi-level data cache.

[0145] The service process processing system and method using multi-level data caching proposed by the present invention can ensure that the service process can read parameter data by setting a multi-level storage scheme of local cache, Redis centralized cache and database, reduce the impact of failures in the production environment on some businesses, and maintain stable operation of the system; the local cache is used as a first-level cache to ensure that the service process can read most of the required parameters from it, avoid frequent reading of the second-level cache or database, and when the Redis centralized cache or database fails to serve, it ensures that most transactions can persist for a period of time without being affected by the failure and proceed smoothly, so that transactions can continue without direct failure, thus achieving high availability; Redis cache is used as a first-level cache to ensure that the service process can read most of the required parameters from it, avoid frequent reading of the second-level cache or database, and when the Redis centralized cache or database fails to serve, it ensures that most transactions can persist for a period of time without being affected by the failure and proceed smoothly, so that transactions can continue without direct failure, thus achieving high availability; It is a secondary cache. If the service process cannot hit data in the local cache unit, it can access the Redis centralized cache unit, and when it is unavailable, the service process can directly read the database; both the Redis cluster and the database have high availability. The Redis cluster mode of the Redis centralized cache adopts the hash slot algorithm and the master-slave replication model to ensure that changes and downtime of the storage node will not affect the external services of the entire cluster; the master-slave multi-copy mode of the database can switch to the database slave node when the database master node is unavailable, which is transparent to the application and enables the application to read and write data from the database without being affected; the overall solution can maintain the stable operation of the system and buy time for fault repair.

[0146] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] The present invention is described with reference to flowcharts and / or block diagrams of methods and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0150] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A service process processing system using multi-level data cache, characterized in that: The system includes: a local cache unit, a Redis centralized cache unit, a database and a service process processing module; wherein, The local cache unit, Redis centralized cache unit, and database use a multi-level data cache mode to store data, with access priorities from high to low as follows: local cache unit, Redis centralized cache unit, and database. The architecture of this system is applied to banking business scenarios. The three-level storage of local cache, Redis centralized cache, and database supports and complements each other at different storage levels. When one or two-level storage components become unavailable, the service process can continue to complete transactions. The service process processing module is used to give priority to accessing the local cache unit when the service process is started; If the data cannot be found in the local cache unit, the Redis centralized cache unit is accessed; if the data is found, the data is read and the transaction processing continues; If the data cannot be found in the Redis centralized cache unit, the database is accessed; if the data is found, the data is read and the transaction processing continues. When the Redis centralized cache unit is unavailable and the service process cannot find the data in the local cache unit, the service process directly reads the data from the database, maintaining availability but reducing transaction performance and concurrency. When the database is in an extreme state where it cannot read or write, some read parameter requests are satisfied through local cache and Redis centralized cache, maintaining availability and partition tolerance based on CAP theory. When both the Redis cluster and the database are unavailable, the local cache supports read data requests for a period of time, supports service process access, and keeps transactions processing.

2. The service process processing system using multi-level data cache according to claim 1, characterized in that: The local cache unit is a first-level cache, which is used to cache data with an access frequency greater than a set value.

3. The service process processing system using multi-level data cache according to claim 2, characterized in that: The Redis centralized cache unit is a secondary cache, which is used to cache a certain amount of data in the database.

4. The service process processing system using multi-level data cache according to claim 3, characterized in that: The system also includes: The data synchronization module is used for regular multi-level data caching. When the data changes, parameter synchronization technology is used to keep the data in the local cache, Redis centralized cache unit and database consistent.

5. The service process processing system using multi-level data cache according to claim 2, characterized in that: The Redis centralized cache unit adopts a cluster mode, and the cluster mode uses hash slots. When the number of hash slots of storage nodes in the Redis centralized cache unit is deleted or changed, the Redis centralized cache unit remains available.

6. The service process processing system using multi-level data cache according to claim 5, characterized in that: In the cluster mode, the Redis centralized cache unit includes multiple Redis master nodes, each Redis master node is correspondingly provided with at least one Redis slave node, and when the Redis master node is inaccessible, access is performed through the Redis slave node.

7. The service process processing system using multi-level data cache according to claim 1, characterized in that: The database adopts a master-slave multi-copy mode, with a database master node and at least one database slave node. Data is synchronized in real time through Binlog; when the database master node is unavailable, it automatically switches to the database slave node.

8. The service process processing system using multi-level data cache according to claim 6, characterized in that: The Redis centralized cache unit includes a preset Redis cluster proxy component and multiple data partitions, each data partition includes multiple Redis master nodes and Redis slave nodes; wherein, When a service process initiates an access request to the Redis centralized cache unit, the target data partition that is not down is determined through the Redis cluster proxy component and the data key value in the access request; the access request is routed to the target data partition, and the data key value is processed to obtain a hash slot number; the data access request is routed to the Redis master node corresponding to the hash slot number in the target data partition. If the Redis master node is inaccessible, access is performed through the Redis slave node.

9. The service process processing system using multi-level data cache according to claim 7, characterized in that: The database is provided with a master-slave synchronization unit; wherein, The master-slave synchronization unit is used to monitor whether a new Binlog file is generated by the database master node; if a new Binlog file is generated, the newly generated Binlog file is packaged and compressed and sent to the database standby node; if no new Binlog file is generated, the monitoring continues; the database standby node unpacks and verifies the received compressed Binlog file, obtains complete data, and saves it to the storage area.

10. A service process processing method using multi-level data cache, characterized in that: The method includes: A multi-level data caching mode is set, wherein the access priority in descending order is: local cache unit, Redis centralized cache unit, and database. The architecture of this multi-level data caching mode is applied in banking business scenarios. The local cache, Redis centralized cache, and database three-level storage support each other at different storage levels. When one or two-level storage components become unavailable, the service process can continue to complete transactions. When the service process starts, it gives priority to accessing the local cache unit; If the data cannot be found in the local cache unit, the Redis centralized cache unit is accessed; if the data is found, the data is read and the transaction processing continues; If the data cannot be found in the Redis centralized cache unit, the database is accessed; if the data is found, the data is read and the transaction processing continues. When the Redis centralized cache unit is unavailable and the service process cannot find the data in the local cache unit, the service process directly reads the data from the database, maintaining availability but reducing transaction performance and concurrency. When the database is in an extreme state where it cannot read or write, some read parameter requests are satisfied through local cache and Redis centralized cache, maintaining availability and partition tolerance based on CAP theory. When both the Redis cluster and the database are unavailable, the local cache supports read data requests for a period of time, supports service process access, and keeps transactions processing.

11. The service process processing method using multi-level data cache according to claim 10, characterized in that: The method further includes: Regular multi-level data caching. When data changes, parameter synchronization technology is used to keep the data in the local cache, Redis centralized cache unit and database consistent.

12. The service process processing method using multi-level data cache according to claim 10, characterized in that: The Redis centralized cache unit adopts a cluster mode, and the cluster mode uses hash slots. When the number of hash slots of storage nodes in the Redis centralized cache unit is deleted or changed, the Redis centralized cache unit remains available.

13. The service process processing method using multi-level data cache according to claim 12, characterized in that: In the cluster mode, the Redis centralized cache unit includes multiple Redis master nodes, each Redis master node is correspondingly provided with at least one Redis slave node, and when the Redis master node is inaccessible, access is performed through the Redis slave node.

14. The service process processing method using multi-level data cache according to claim 10, characterized in that: The database adopts a master-slave multi-copy mode, with a database master node and at least one database slave node. Data is synchronized in real time through Binlog; when the database master node is unavailable, it automatically switches to the database slave node.

15. The service process processing method using multi-level data cache according to claim 13, characterized in that: The Redis centralized cache unit includes a preset Redis cluster proxy component and multiple data partitions, each data partition includes multiple Redis master nodes and Redis slave nodes; wherein, When a service process initiates an access request to the Redis centralized cache unit, the target data partition that is not down is determined through the Redis cluster proxy component and the data key value in the access request; the access request is routed to the target data partition, and the data key value is processed to obtain a hash slot number; the data access request is routed to the Redis master node corresponding to the hash slot number in the target data partition. If the Redis master node is inaccessible, access is performed through the Redis slave node.

16. The service process processing method using multi-level data cache according to claim 14, characterized in that: The database is provided with a master-slave synchronization unit; wherein, The master-slave synchronization unit is used to monitor whether a new Binlog file is generated by the database master node; if a new Binlog file is generated, the newly generated Binlog file is packaged and compressed and sent to the database standby node; if no new Binlog file is generated, the monitoring continues; the database standby node unpacks and verifies the received compressed Binlog file, obtains complete data, and saves it to the storage area.

17. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 10 to 16 is implemented.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 10 to 16 is implemented.

Images