Cache data semi-synchronization method

Abstract

The present application relates to the technical field of middleware, and in particular to a cache data semi-synchronization method, comprising comprises the following steps: writing and running a distributed cache proxy layer service using a preset programming language, receiving a read / write command request from an application client, parsing a Key consistent hashing value and sending same to a rear-end Redis sharding node. A master instance parses a message packet, marks a write operation and records same in an AOF file. Upon write to disk, the master instance sends an AOF file transfer command to a slave instance node, and the slave instance acknowledges reception. When receiving a reception response, the master instance updates an acknowledgment offset of the slave node and responds to the client. When network delay is high, data of a master node and a slave node is marked as "asynchronous", and it is automatically downgraded to asynchronous replication. According to the present application, by means of a self-developed Redis kernel, an asynchronous ack mechanism is used to optimize redis to achieve master-slave semi-synchronization, thereby enhancing data reliability of a single point of failure, and eliminating the influence introduced by a native Redis asynchronous replication mechanism such as data delay and data consistency.

Description

A method for semi-synchronization of cached data

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on December 14, 2023, with application number 202311720255.0 and invention name “A method for semi-synchronization of cached data”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application belongs to the technical field of middleware, and in particular relates to a method for semi-synchronization of cached data. Background Art

[0004] With the rapid development of computer technology, cache data synchronization has become an important means to improve data processing efficiency and ensure data consistency. However, existing cache data synchronization methods have some problems. As shown in Figure 1, the process has the following problems:

[0005] Caching master-slave data delay. During the master-slave replication process, due to resource constraints such as server network, hardware failure, and high host load, data replication between master and slave nodes may be delayed.

[0006] The cache master-slave data consistency problem. During the master-slave replication process, the data replication delay problem based on the above scenario, in the read-write separation architecture scenario, especially the application with high real-time requirements and low latency, will cause the data read by the business from the slave node to be inconsistent with the actual data, which may affect the business logic and fail to work as expected.

[0007] Regarding the problem of fault recovery, based on the above scenario, when master-slave replication experiences master-slave data delays or even master-slave data inconsistencies, when the master node crashes abnormally and a master-slave failover is required, the inconsistency of master-slave node data will cause the slave node to lose data that has not yet been synchronized. When the old slave node switches to the new master node, the data loss caused by the inconsistency of master-slave replication will bring unpredictable risks to the business. Summary of the Invention

[0008] In view of the above shortcomings of the existing technology, the purpose of this application is to provide a method for semi-synchronous caching of data. In order to address the data delay, data consistency and other effects brought by the native Redis asynchronous replication mechanism, the asynchronous ack mechanism is used to deeply customize and optimize the native Redis master-slave replication process. Through the incremental command semi-synchronous replication mechanism, the data reliability of single point failures is greatly improved. Through a customized semi-synchronous degradation strategy, it can automatically downgrade to asynchronous replication in the event of high network fluctuations and delays, taking into account the throughput performance of the Redis instance in a scenario with high data reliability.

[0009] This application proposes a method for semi-synchronous caching of data, comprising the following steps:

[0010] S1: Use the preset programming language to write and run the distributed cache proxy layer service Access;

[0011] S2: Receives read and write command requests from the application client in Access, parses the key consistent hash value in the message, and sends it to the corresponding Redis shard node in the backend;

[0012] S3: The master instance parses the message. When it receives a write command from a client, it marks the client and records the offset of the corresponding write command in the AOF file.

[0013] S4: The master instance also appends the write operation command to the AOF file and writes it to the disk.

[0014] S5: After the write operation command is written to the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node;

[0015] S6: After receiving the AOF file transfer command from the master node, the slave instance executes the command and signs for it;

[0016] S7: After the master instance receives the receipt response command from the slave instance, it updates the confirmation offset of the slave node and responds to the client with an event reply.

[0017] S8: When the network latency between the master and slave nodes is high, the Redis master and slave data synchronization progress is not equal and the master and slave data synchronization progress is stagnant. If the stagnation time is greater than the preset threshold, the current master and slave node data is marked as "out of sync";

[0018] S9: When the master-slave data synchronization status is "out of sync", the master-slave replication mechanism is automatically downgraded from a semi-synchronous mechanism to an asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronous", and the asynchronous replication mechanism is automatically restored to a semi-synchronous mechanism.

[0019] Optionally, the threshold is 1 minute.

[0020] Optionally, the preset programming language is Java.

[0021] Optionally, the distributed cache is NoSQL in-memory database software.

[0022] Optionally, the NoSQL in-memory database software is compatible with the Redis protocol.

[0023] Optionally, the marking includes establishing different data synchronization channels between the master and slave nodes.

[0024] Optionally, the AOF file is a command log file.

[0025] Optionally, the AOF file also includes the timestamp and operation content information of the write operation command.

[0026] Optionally, the downgrading further includes caching part of the data on a local hard disk.

[0027] Optionally, the master instance further includes writing the current timestamp into the confirmation offset of the slave node.

[0028] The beneficial effects of this application are as follows:

[0029] This application uses the self-developed Redis kernel and the asynchronous ack mechanism to optimize Redis to achieve master-slave semi-synchronization, improve data reliability in the event of single point failure, and solve the data delay, data consistency and other impacts brought by the native Redis asynchronous replication mechanism. At the same time, it adds a custom strategy configuration for semi-synchronous downgrade. In response to the impact of network fluctuations or high latency between master and slave nodes on overall throughput performance, the semi-synchronous replication strategy is automatically downgraded to asynchronous replication, and then automatically restored to the semi-synchronous replication strategy when the network is restored. At the same time, a new Redis command, Status, is added to judge the synchronization status of Redis master-slave data in real time. It is more friendly to external monitoring systems and can monitor the synchronization status of Redis master-slave data in real time, which is progressive and advanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the present application. Throughout the drawings, the same reference numerals represent the same components. Obviously, the drawings described below are only some of the embodiments described in the present application. Those skilled in the art can also obtain other drawings based on these drawings.

[0031] FIG1 is a schematic diagram of a cache data synchronization method in the prior art;

[0032] FIG2 is a flowchart of the native Redis data asynchronous replication in an embodiment of the present application;

[0033] FIG3 is a flow chart of a distributed cache semi-synchronization mechanism according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without making creative work should fall within the scope of protection of this application.

[0035] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in this application.

[0036] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of methods and systems consistent with certain aspects of the present application, as detailed in the appended claims.

[0037] This application proposes a method for semi-synchronous caching of data to solve the problems of data delay, data consistency, and fault recovery caused by the native Redis asynchronous replication mechanism.

[0038] In order to assist personnel in understanding this application, the following nouns appearing in the text are explained:

[0039] Distributed cache is a NoSQL memory database software compatible with the Redis protocol. It can accept data query and storage requests from other client applications, and transmit them to the backend cache storage shard nodes through the proxy middleware consistent hash sharding and respond.

[0040] Asynchronous Data Synchronization: Native Redis uses an asynchronous replication mechanism for data synchronization. In master-slave replication, the master node is responsible for write operations, and the slave nodes asynchronously receive updated data pulled from the master. In scenarios requiring high data real-time performance, data on the slave nodes may lag behind. If the master node fails, data loss can result, leading to data consistency issues.

[0041] Data semi-synchronization: This mechanism is a cross between synchronous and asynchronous data replication, designed to ensure data consistency while minimizing the impact on write performance. In semi-synchronization, after a write operation completes on the master node, the slave node must confirm receipt of the write operation before the master node responds to the client and proceeds with the next operation. Meanwhile, the slave node writes to disk. This mechanism can reduce data latency to a certain extent and provide higher data assurance.

[0042] The master-slave data synchronization in native Redis is asynchronous replication. Under some abnormal circumstances, it will bring a series of problems to the business, such as master-slave data delay, master-slave data consistency issues, and failure scenario recovery. The process of native Redis master-slave data asynchronous replication is as follows:

[0043] The original solution is shown in Figure 2:

[0044] 1. When the client connects to the Redis master instance, it sends a write operation command to the master node.

[0045] 2. The master instance parses the message and, upon receiving a write operation command from the client, executes the write operation command to update the key-value data in memory.

[0046] 3. The master instance appends the write operation command to the AOF file (if the AOF function is enabled).

[0047] 4. The master instance writes the write operation command to the local master-slave replication buffer and responds to the client with a response event.

[0048] 5. The master instance continues to receive other read and write operation commands from the client, while the asynchronous background thread sends the commands in the replication buffer to the slave node.

[0049] 6. The slave node receives the replication command, executes it, signs it, and responds to the master node.

[0050] 7. The master instance updates the replication offset in the replication buffer, completing the asynchronous master-slave replication process.

[0051] However, the master-slave data synchronization of the above native Redis is asynchronous replication, which may cause a series of problems to the business under some abnormal circumstances, such as master-slave data delay, master-slave data consistency issues, and fault scenario recovery.

[0052] The specific questions are as follows:

[0053] Scenario 1. Cache master-slave data delay

[0054] During the master-slave replication process, due to resource constraints such as server network, hardware failure, and high host load, data replication between master and slave nodes may be delayed.

[0055] Scenario 2: Cache master-slave data consistency issue

[0056] During the master-slave replication process, the data replication delay problem based on scenario 1, in the read-write separation architecture scenario, especially in applications with high real-time requirements and low latency, will cause the data read from the slave node by the business to be inconsistent with the actual data, which may affect the business logic and prevent it from working as expected.

[0057] Scenario 3: Fault recovery issues

[0058] Based on the above scenario, when master-slave replication experiences master-slave data delays or even master-slave data inconsistencies, and when the master node crashes abnormally and a master-slave failover is required, the inconsistency in master-slave node data will cause the slave node to lose data that has not yet been synchronized. When the old slave node switches to the new master node, the data loss caused by the inconsistency in master-slave replication will bring unpredictable risks to the business.

[0059] In order to solve the existing problems, in this application, as shown in FIG3 , the following implementation method is proposed:

[0060] Implementation Method 1

[0061] A method for semi-synchronizing cached data includes the following steps:

[0062] S1: Use the preset programming language to write and run the distributed cache proxy layer service Access;

[0063] S2: Receives read and write command requests from the application client in Access, parses the key consistent hash value in the message, and sends it to the corresponding Redis shard node in the backend;

[0064] S3: The master instance parses the message. When it receives a write command from a client, it marks the client and records the offset of the corresponding write command in the AOF file.

[0065] S4: The master instance also appends the write operation command to the AOF file and writes it to the disk.

[0066] S5: After the write operation command is written to the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node;

[0067] S6: After receiving the AOF file transfer command from the master node, the slave instance executes the command and signs for it;

[0068] S7: After the master instance receives the receipt response command from the slave instance, it updates the confirmation offset of the slave node and responds to the client with an event reply.

[0069] S8: When the network latency between the master and slave nodes is high, the Redis master and slave data synchronization progress is not equal and the master and slave data synchronization progress is stagnant. If the stagnation time is greater than the preset threshold, the current master and slave node data is marked as "out of sync";

[0070] S9: When the master-slave data synchronization status is "out of sync", the master-slave replication mechanism is automatically downgraded from a semi-synchronous mechanism to an asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronous", and the asynchronous replication mechanism is automatically restored to a semi-synchronous mechanism.

[0071] The marking includes establishing different data synchronization channels between the master and slave nodes.

[0072] The AOF file is a log file of commands.

[0073] The AOF file also includes the timestamp and operation content information of the write operation command.

[0074] The downgrading further includes caching part of the data on a local hard disk.

[0075] The master instance also includes writing the current timestamp into the confirmation offset of the slave node.

[0076] Implementation Method 2

[0077] A method for semi-synchronizing cached data includes the following steps:

[0078] S1: Use the preset programming language to write and run the distributed cache proxy layer service Access;

[0079] S2: Receives read and write command requests from the application client in Access, parses the key consistent hash value in the message, and sends it to the corresponding Redis shard node in the backend;

[0080] S3: The master instance parses the message. When it receives a write command from a client, it marks the client and records the offset of the corresponding write command in the AOF file.

[0081] S4: The master instance also appends the write operation command to the AOF file and writes it to the disk.

[0082] S5: After the write operation command is written to the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node;

[0083] S6: After receiving the AOF file transfer command from the master node, the slave instance executes the command and signs for it;

[0084] S7: After the master instance receives the receipt response command from the slave instance, it updates the confirmation offset of the slave node and responds to the client with an event reply.

[0085] S8: When the network latency between the master and slave nodes is high, the Redis master and slave data synchronization progress is not equal and the master and slave data synchronization progress is stagnant. The stagnation time is greater than the preset threshold, which is 1 minute. The current master and slave node data is marked as "out of sync";

[0086] S9: When the master-slave data synchronization status is "out of sync", the master-slave replication mechanism is automatically downgraded from a semi-synchronous mechanism to an asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronous", and the asynchronous replication mechanism is automatically restored to a semi-synchronous mechanism.

[0087] The marking includes establishing different data synchronization channels between the master and slave nodes.

[0088] The AOF file is a log file of commands.

[0089] The AOF file also includes the timestamp and operation content information of the write operation command.

[0090] The downgrading further includes caching part of the data on a local hard disk.

[0091] The master instance also includes writing the current timestamp into the confirmation offset of the slave node.

[0092] Implementation 3

[0093] A method for semi-synchronizing cached data includes the following steps:

[0094] S1: Use the default programming language to write and run the distributed cache proxy layer service Access. The distributed cache is a NoSQL in-memory database software.

[0095] S2: Receives read and write command requests from the application client in Access, parses the key consistent hash value in the message, and sends it to the corresponding Redis shard node in the backend;

[0096] S3: The master instance parses the message. When it receives a write command from a client, it marks the client and records the offset of the corresponding write command in the AOF file.

[0097] S4: The master instance also appends the write operation command to the AOF file and writes it to the disk.

[0098] S5: After the write operation command is written to the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node;

[0099] S6: After receiving the AOF file transfer command from the master node, the slave instance executes the command and signs for it;

[0100] S7: After the master instance receives the receipt response command from the slave instance, it updates the confirmation offset of the slave node and responds to the client with an event reply.

[0101] S8: When the network latency between the master and slave nodes is high, the Redis master and slave data synchronization progress is not equal and the master and slave data synchronization progress is stagnant. The stagnation time is greater than the preset threshold, which is 1 minute. The current master and slave node data is marked as "out of sync";

[0102] S9: When the master-slave data synchronization status is "out of sync", the master-slave replication mechanism is automatically downgraded from a semi-synchronous mechanism to an asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronous", and the asynchronous replication mechanism is automatically restored to a semi-synchronous mechanism.

[0103] Implementation 4

[0104] A method for semi-synchronizing cached data includes the following steps:

[0105] S1: Use the preset programming language to write and run the distributed cache proxy layer service Access. The distributed cache is NoSQL in-memory database software, which is compatible with the Redis protocol.

[0106] S2: Receives read and write command requests from the application client in Access, parses the key consistent hash value in the message, and sends it to the corresponding Redis shard node in the backend;

[0107] S3: The master instance parses the message. When it receives a write command from a client, it marks the client and records the offset of the corresponding write command in the AOF file.

[0108] S4: The master instance also appends the write operation command to the AOF file and writes it to the disk.

[0109] S5: After the write operation command is written to the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node;

[0110] S6: After receiving the AOF file transfer command from the master node, the slave instance executes the command and signs for it;

[0111] S7: After the master instance receives the receipt response command from the slave instance, it updates the confirmation offset of the slave node and responds to the client with an event reply.

[0112] S8: When the network latency between the master and slave nodes is high, the Redis master and slave data synchronization progress is not equal and the master and slave data synchronization progress is stagnant. The stagnation time is greater than the preset threshold, which is 1 minute. The current master and slave node data is marked as "out of sync";

[0113] S9: When the master-slave data synchronization status is "out of sync", the master-slave replication mechanism is automatically downgraded from a semi-synchronous mechanism to an asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronous", and the asynchronous replication mechanism is automatically restored to a semi-synchronous mechanism.

[0114] Implementation 5

[0115] 1. Write and run the distributed cache proxy layer service Access

[0116] First, use a pre-defined programming language (such as Java) to write and run the distributed cache proxy layer service Access. Access is responsible for receiving read and write command requests from application clients, parsing the key consistent hash value in the message, and sending it to the corresponding Redis shard node on the backend.

[0117] 2. The master instance parses the message and marks the write operation command

[0118] When the master instance receives a write command from a client, it marks the client and records the offset of the corresponding write command in the AOF file. In this way, when the slave instance needs to synchronize data, it can obtain the write command from the AOF file.

[0119] 3. The master instance appends the write operation command to the AOF file and writes it to the disk

[0120] The master instance also appends the write command to the AOF file and writes it to disk. This step depends on the node's currently configured storage policy. After the write command is stored in the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave node.

[0121] 4. Execute commands from the instance and sign for them

[0122] After receiving the AOF file transfer command from the master, the slave instance executes the command and signs for it. At this point, the slave sends a sign-off response command to the master before writing the data to disk. This way, the master instance knows that the slave instance has successfully received and executed the write operation command.

[0123] 5. Update the confirmation offset of the slave node and reply to the client with the response event

[0124] When the master instance receives the slave instance's signature response command, it updates the slave node's confirmation offset and responds to the client with an acknowledgement event. This way, the client knows that the write operation command has been successfully executed.

[0125] 6. Mark the master-slave node data as "out of sync" and downgrade the replication mechanism

[0126] When network latency between the master and slave nodes is high, the Redis master-slave data synchronization progress fails to catch up and stalls. If the stall lasts longer than a preset threshold (e.g., 1 minute), the master-slave data is marked as "out of sync." In this case, the master-slave replication mechanism is automatically downgraded from semi-synchronous to asynchronous. When the network between the master and slave nodes is restored, the master-slave data status is updated to "in sync," and the asynchronous replication mechanism is automatically restored to semi-synchronous.

[0127] 7. Application Scenarios and Advantages

[0128] The method for semi-synchronization of cache data provided in the embodiment of the present application can be applied to various scenarios that require a distributed cache system, such as e-commerce systems, social applications, etc. By using a consistent hashing algorithm for cache proxy and a master-slave replication mechanism for data synchronization, the efficiency and reliability of the cache can be effectively improved. At the same time, by automatically detecting network delays and stagnant replication progress, data asynchrony problems can be avoided, improving the availability and stability of the system. In addition, the embodiment of the present application can also be used in combination with other cache synchronization methods to improve data reliability and consistency.

[0129] Based on the same inventive concept, another embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus.

[0130] Memory for storing computer programs;

[0131] The processor is used to implement the cache data semi-synchronization method of the present application when executing the program stored in the memory.

[0132] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the above terminal and other devices. The memory can include a random access memory (RAM) or a non-volatile memory, such as at least one disk storage. Optionally, the memory can also be at least one storage system located away from the aforementioned processor.

[0133] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0134] In addition, to achieve the above-mentioned purpose, an embodiment of the present application further proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for semi-synchronization of cache data of an embodiment of the present application.

[0135] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of the present application may take the form of a computer program product implemented on one or more computer-usable vehicles (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] The present application embodiment is described with reference to the flow chart and / or block diagram according to the present application embodiment method, terminal equipment (system) and computer program product.It should be understood that each flow process and / or box in the flow chart and / or block diagram and the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer, special-purpose computer, embedded processing machine or other programmable data processing terminal equipment to produce a machine, so that the instruction executed by the processor of computer or other programmable data processing terminal equipment produces the system for realizing the function specified in flow chart one flow chart or multiple flow charts and / or block diagram one block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction system that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0139] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. "And / or" means that either one of the two can be selected, or both can be selected. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or terminal device that includes the elements.

[0140] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0141] In summary, the embodiments of the present application provide a method for semi-synchronization of cache data that can effectively implement functions such as proxy and sharding of cache data, data synchronization and replication mechanism management, improve the efficiency and reliability of the distributed cache system, and provide an effective solution for related applications.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of this application, and are not intended to limit them. Although this application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and these modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of this application. Any changes or replacements that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered by the scope of protection of this application.

Claims

1. A method for semi-synchronization of cache data, characterized in that: The steps include: S1: Use the preset programming language to write and run the distributed cache proxy layer service Access; S2: Receives read and write command requests from the application client in Access, parses the Key consistent hash value in the message, and sends it to the corresponding Redis shard node in the backend; S3: The main instance parses the message. When receiving a write operation command from the client, it marks the client and records the offset of the corresponding write command in the AOF file. S4: The master instance also appends the write operation command to the AOF file and writes it to the disk; S5: After the write operation command is written to the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node; S6: After receiving the AOF file transfer command from the master node, the slave instance executes the command and signs for it; S7: After the master instance receives the receipt response command from the slave instance, it updates the confirmation offset of the slave node and responds to the event reply client; S8: When the network delay between the master and slave nodes is high, the Redis master-slave data synchronization progress is not equal and the master-slave data synchronization progress is stagnant. The stagnation time is greater than the preset threshold, which means that the current master-slave node data is marked as "out of sync"; S9: When the master-slave data synchronization status is "out of sync", the master-slave replication mechanism is automatically downgraded from a semi-synchronous mechanism to an asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronous", and the asynchronous replication mechanism is automatically restored to a semi-synchronous mechanism.

2. The method for semi-synchronization of cache data according to claim 1, characterized in that: In S8, the threshold is 1 minute.

3. The method for semi-synchronization of cache data according to claim 1, characterized in that: The preset programming language is Java.

4. The method for semi-synchronization of cache data according to claim 1, characterized in that: The distributed cache is a NoSQL in-memory database software.

5. The method for semi-synchronization of cache data according to claim 4, characterized in that: The NoSQL in-memory database software is compatible with the Redis protocol.

6. The method for semi-synchronization of cache data according to claim 1, characterized in that: The marking includes establishing different data synchronization channels between the master and slave nodes.

7. The method for semi-synchronization of cache data according to claim 1, characterized in that: The AOF file is a log file of the command.

8. The method for semi-synchronization of cache data according to claim 7, characterized in that: The AOF file also includes the timestamp and operation content information of the write operation command.

9. The method for semi-synchronization of cache data according to claim 1, characterized in that: The downgrading further includes caching part of the data on a local hard disk.

10. The method for semi-synchronization of cache data according to claim 8, characterized in that: The master instance also includes writing the current timestamp into the confirmation offset of the slave node.

Images