A method for semi-synchronous caching data

Abstract

The present invention relates to a method for semi-synchronous caching data, belonging to the technical field of middleware, and includes the following steps: Write and run a distributed cache proxy layer service using a preset programming language, receive read and write command requests from application clients, parse the Key consistent hash value, and send it to the backend Redis shard nodes. The master instance parses the message packet, marks the write operation and records it in the AOF file. After the disk write is completed, the master instance sends an AOF file transfer command to the slave instance node, and the slave instance executes the receipt. After the master instance receives the receipt response, it updates the confirmation offset of the slave node and responds to the client. When the network latency is high, mark the data of the master and slave nodes as "out of sync" and automatically degrade to asynchronous replication. The present invention optimizes the implementation of master-slave semi-synchronization in Redis by self-developing the Redis kernel and using the asynchronous ack mechanism, improves the data reliability of single-point failures, and solves the problems of data latency and data consistency caused by the native Redis asynchronous replication mechanism.

Description

Technical Field

[0001] The present invention belongs to the technical field of middleware, and in particular relates to a method for semi-synchronization of cache data. Background Art

[0002] With the rapid development of computer technology, cache data synchronization has become an important means to improve data processing efficiency and ensure data consistency. However, existing cache data synchronization methods have some problems, such as Figure 2 As shown, the process problems are as follows:

[0003] Caching master-slave data delay. During the master-slave replication process, due to resource constraints such as server network, hardware failure, and high host load, data replication between master and slave nodes may be delayed.

[0004] The cache master-slave data consistency problem. During the master-slave replication process, the data replication delay problem based on the above scenario, in the read-write separation architecture scenario, especially the application with high real-time requirements and low latency, will cause the data read by the business from the slave node to be inconsistent with the actual data, which may affect the business logic and fail to work as expected.

[0005] Regarding the problem of fault recovery, based on the above scenario, when master-slave replication experiences master-slave data delays or even master-slave data inconsistencies, when the master node crashes abnormally and a master-slave failover is required, due to the inconsistency of master-slave node data, data that the slave node has not yet synchronized will be lost. When the old slave node switches to the new master node, the data loss caused by the inconsistency of master-slave replication will bring unpredictable risks to the business. Summary of the Invention

[0006] In view of the above shortcomings of the existing technology, the purpose of the invention is to provide a method for semi-synchronous caching of data. In order to address the data delay, data consistency and other impacts brought by the native Redis asynchronous replication mechanism, the asynchronous ack mechanism is used to deeply customize and optimize the native Redis master-slave replication process. Through the incremental command semi-synchronous replication mechanism, the data reliability of single point failures is greatly improved. Through the customized semi-synchronous degradation strategy, it can automatically downgrade to asynchronous replication in the case of high network fluctuations and delays, taking into account the throughput performance of the Redis instance in the scenario of high data reliability.

[0007] The present invention proposes a method for semi-synchronous caching of data, comprising the following steps:

[0008] S1: Use the preset programming language to write and run the distributed cache proxy layer service Access;

[0009] S2: The Access receives the read / write command request from the application client, parses the Key consistent hash value in the message, and sends it to the corresponding Redis shard node at the backend;

[0010] S3: The master instance parses the message packet. When receiving the write operation command from the client, it marks the client and records the offset of the corresponding write command in the AOF file;

[0011] S4: And the master instance simultaneously appends and writes the write operation command to the AOF file for disk writing;

[0012] S5: After the write operation command is written to the AOF file for disk, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node;

[0013] S6: After the slave instance receives the AOF file transfer command from the master node, it executes the command and signs for receipt;

[0014] S7: When the master instance receives the receipt response command from the slave instance, it updates the confirmation offset of the slave node and replies to the event to respond to the client;

[0015] S8: When the network latency between the master and slave nodes is relatively high, the Redis master-slave data synchronization progress has not caught up and the master-slave data synchronization progress has stalled, and the stall time is greater than the preset threshold, then mark the current master-slave node data as "out of sync" status;

[0016] S9: When the master-slave data synchronization status is "out of sync", automatically downgrade the master-slave replication mechanism from the semi-synchronous mechanism to the asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronized", and the asynchronous replication mechanism is automatically restored to the semi-synchronous mechanism again.

[0017] Further, the threshold is 1 minute.

[0018] Further, the preset programming language is Java.

[0019] Further, the distributed cache is a NoSQL in-memory database software.

[0020] Further, the NoSQL in-memory database software is compatible with the Redis protocol.

[0021] Further, the marking includes establishing different data synchronization channels between the master and slave nodes.

[0022] Further, the AOF file is a log file of commands.

[0023] Further, the AOF file also includes the timestamp and operation content information of the write operation command.

[0024] Furthermore, the downgrading further includes caching part of the data on a local hard disk.

[0025] Furthermore, the master instance further includes writing the current timestamp into the confirmation offset of the slave node.

[0026] The beneficial effects of the present invention are as follows:

[0027] The present invention uses the self-developed Redis kernel and the asynchronous ack mechanism to optimize Redis to achieve master-slave semi-synchronization, improve data reliability of single point failures, and solve the data delay, data consistency and other effects brought by the native Redis asynchronous replication mechanism. At the same time, it adds a custom strategy configuration for semi-synchronous downgrade. In response to the impact of network fluctuations or high latency between master and slave nodes on overall throughput performance, the semi-synchronous replication strategy is automatically downgraded to asynchronous replication, and then automatically restored to the semi-synchronous replication strategy when the network is restored. At the same time, a new Redis command Status command is added to judge the synchronization status of Redis master-slave data in real time. It is more friendly to external monitoring systems and can monitor the synchronization status of Redis master-slave data in real time, which is progressive and advanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. Throughout the drawings, the same reference numerals represent the same components. Obviously, the drawings described below are only some of the embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings.

[0029] Figure 1 This is a flow chart of a distributed cache semi-synchronization mechanism according to an embodiment of the present invention;

[0030] Figure 2 This is a flowchart of the native Redis data asynchronous replication in an embodiment of the present invention;

[0031] Figure 3 This is a flow chart of the distributed cache semi-synchronization mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the scope of protection of the present invention.

[0033] In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in the present invention.

[0034] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of methods and systems consistent with some aspects of the present invention as detailed in the appended claims.

[0035] The present invention proposes a method for semi-synchronous caching data to solve the problems of data latency, data consistency, and fault recovery brought by the native Redis asynchronous replication mechanism.

[0036] To assist personnel in understanding the present invention, explanations are made for the nouns appearing in the text:

[0037] Distributed cache is a NoSQL in-memory database software compatible with the Redis protocol. It can accept data query and storage requests from other client applications, and transparently transmit them to the backend cache storage shard nodes through the proxy middleware's consistent hashing sharding and respond.

[0038] Data asynchronous synchronization: The data synchronization of native Redis is an asynchronous replication mechanism. In master-slave replication, the master node is responsible for write operations, and the slave node will asynchronously receive the updated data pulled from the master node. In some scenarios with high requirements for data real-time performance, it may cause the data on the slave node to lag behind. If the master node fails, it will cause data loss and lead to data consistency problems.

[0039] Data semi-synchronization: Data semi-synchronization is a data synchronization mechanism between synchronous and asynchronous data replication, aiming to reduce the impact on write performance while ensuring data consistency. In data semi-synchronization, after the write operation is completed on the master node, it is required that the slave node confirm receiving the message of the write operation, and then the master node responds to the client to continue with the next operation, and at the same time, the slave node writes to disk. This mechanism can reduce data latency to a certain extent and provide higher data assurance.

[0040] The master-slave data synchronization of native Redis is asynchronous replication. In some abnormal situations, it will bring a series of problems to the business, such as master-slave data latency, master-slave data consistency problems, fault scenario recovery, etc. The main process of the native Redis master-slave data asynchronous replication is as follows:

[0041] The original solutions such as Figure 1 and Figure 3 are shown as:

[0042] 1. When the client connects to the Redis master instance, it sends a write operation command to the master node.

[0043] 2. The master instance parses the message and, upon receiving a write operation command from the client, executes the write operation command to update the key-value data in memory.

[0044] 3. The master instance updates and appends the write operation command to the AOF file (if the AOF function is enabled).

[0045] 4. The master instance writes the write operation command to the local master-slave replication buffer and responds to the client with a response event.

[0046] 5. The master instance continues to receive other read and write operation commands from the client, while the asynchronous background thread sends the commands in the replication buffer to the slave node.

[0047] 6. Receive the replication command from the slave node, execute and sign, and respond to the master node

[0048] 7. The master instance updates the replication offset in the replication buffer, completing the asynchronous master-slave replication process.

[0049] However, the master-slave data synchronization of the above native Redis is asynchronous replication, which may cause a series of problems to the business under some abnormal circumstances, such as master-slave data delay, master-slave data consistency issues, and fault scenario recovery.

[0050] The specific questions are as follows:

[0051] Scenario 1. Cache master-slave data delay

[0052] During the master-slave replication process, due to resource constraints such as server network, hardware failure, and high host load, data replication between master and slave nodes may be delayed.

[0053] Scenario 2: Cache master-slave data consistency issue

[0054] During the master-slave replication process, the data replication delay problem based on scenario 1, in the read-write separation architecture scenario, especially in applications with high real-time requirements and low latency, will cause the data read from the slave node by the business to be inconsistent with the actual data, which may affect the business logic and prevent it from working as expected.

[0055] Scenario 3: Fault Recovery Issues

[0056] Based on the above scenarios, when there is a delay in master-slave data replication or even a master-slave data inconsistency, and when the master node crashes abnormally and a master-slave switchover for failover is required, due to the inconsistency of the master-slave node data, the data that the slave node has not had time to synchronize will be lost. When the old slave node is switched to run as the new master node, the data loss caused by the master-slave replication inconsistency will bring unpredictable risks to the business.

[0057] To solve the existing problems, in the present invention, as Figure 2 shown, the following implementation manners are proposed:

[0058] Implementation Manner 1

[0059] A method for semi-synchronous caching data, comprising the following steps:

[0060] S1: Write and run a distributed cache proxy layer service Access using a preset programming language;

[0061] S2: When Access receives read and write command requests from application clients, parse the Key consistent hash value in the message and send it to the corresponding Redis shard node at the backend;

[0062] S3: Parse the message packet through the master instance. When receiving a write operation command from the client, mark the client and record the offset of the corresponding write command in the AOF file;

[0063] S4: And the master instance simultaneously appends the write operation command to the AOF file for disk writing;

[0064] S5: After the write operation command is written to the AOF file for disk writing, the Redis master instance sends a corresponding AOF file transfer command to the peer Redis slave instance node;

[0065] S6: After the slave instance receives the AOF file transfer command from the master node, execute the command and sign for it;

[0066] S7: When the master instance receives the signed response command from the slave instance, update the confirmation offset of the slave node and reply to the event to respond to the client;

[0067] S8: When the network latency between the master and slave nodes is relatively high, the Redis master-slave data synchronization progress has not caught up and the master-slave data synchronization progress has stalled, and the stall time is greater than a preset threshold, then mark the current master-slave node data as "out of sync" status;

[0068] S9: When the master-slave data synchronization status is "out of sync", automatically downgrade the master-slave replication mechanism from the semi-synchronous mechanism to the asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronized", and the asynchronous replication mechanism is automatically restored to the semi-synchronous mechanism again.

[0069] The marking includes establishing different data synchronization channels between the master and slave nodes.

[0070] The AOF file is a log file of commands.

[0071] The AOF file also includes the timestamp and operation content information of the write operation command.

[0072] The downgrading also includes caching some data on the local hard disk.

[0073] The master instance also includes writing the current timestamp into the confirmation offset of the slave node.

[0074] Embodiment 2

[0075] A method for semi-synchronous caching of data, comprising the following steps:

[0076] S1: Write and run the distributed cache proxy layer service Access using a preset programming language;

[0077] S2: When Access receives read and write command requests from the application client, parse the Key consistent hash value in the message and send it to the corresponding Redis shard node at the backend;

[0078] S3: Parse the message packet through the master instance. When receiving the write operation command from the client, mark the client and record the offset of the corresponding write command in the AOF file;

[0079] S4: And the master instance simultaneously appends and writes the write operation command to the AOF file for disk storage;

[0080] S5: After the write operation command is stored on the disk in the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the Redis slave instance node on the other end;

[0081] S6: After the slave instance receives the AOF file transfer command from the master node, execute the command and sign for it;

[0082] S7: When the master instance receives the signed response command from the slave instance, update the confirmation offset of the slave node and reply to the event to respond to the client;

[0083] S8: When the network latency between the master and slave nodes is relatively high, the Redis master-slave data synchronization progress is not caught up and the master-slave data synchronization progress stalls, and the stall time is greater than the preset threshold, and the threshold is 1 minute, that is, mark the current master-slave node data as "out of sync" status;

[0084] S9: When the master-slave data synchronization status is "out of sync", automatically downgrade the master-slave replication mechanism from the semi-synchronous mechanism to the asynchronous replication mechanism. When the network between the master and slave nodes is restored, update the master-slave data status to "synchronized" and automatically restore the asynchronous replication mechanism to the semi-synchronous mechanism again.

[0085] The marking includes establishing different data synchronization channels between the master and slave nodes.

[0086] The AOF file is a log file of commands.

[0087] The AOF file also includes the timestamp of the write operation command and the operation content information.

[0088] The downgrading also includes caching some data to the local hard disk.

[0089] The master instance also includes writing the current timestamp into the confirmation offset of the slave node.

[0090] Embodiment 3

[0091] A method for semi-synchronous caching data, comprising the following steps:

[0092] S1: Write and run the distributed cache proxy layer service Access using a preset programming language. The distributed cache is a NoSQL in-memory database software;

[0093] S2: When Access receives read and write command requests from the application client, parse the Key consistent hash value in the message and send it to the corresponding Redis shard node at the back end;

[0094] S3: Parse the message packet through the master instance. When receiving a write operation command from the client, mark the client and record the offset of the corresponding write command in the AOF file;

[0095] S4: And the master instance simultaneously appends and writes the write operation command to the AOF file for disk storage;

[0096] S5: After the write operation command is stored on the disk in the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node;

[0097] S6: After the slave instance receives the AOF file transfer command from the master node, execute the command and sign for it;

[0098] S7: When the master instance receives the signed response command from the slave instance, update the confirmation offset of the slave node and reply to the event to respond to the client;

[0099] S8: When the network latency between the master and slave nodes is high, the Redis master-slave data synchronization progress has not caught up and the master-slave data synchronization progress has stalled, and the stall time is greater than a preset threshold, which is 1 minute, that is, mark the current master-slave node data as "out of sync" state;

[0100] S9: When the master-slave data synchronization state is "out of sync", automatically downgrade the master-slave replication mechanism from the semi-synchronous mechanism to the asynchronous replication mechanism. When the network between the master and slave nodes resumes, the master-slave data state is updated to "synchronized", and the asynchronous replication mechanism is automatically restored to the semi-synchronous mechanism again.

[0101] Embodiment 4

[0102] A method for semi-synchronous caching data, comprising the following steps:

[0103] S1: Write and run a distributed cache proxy layer service Access using a preset programming language. The distributed cache is a NoSQL in-memory database software, and the NoSQL in-memory database software is compatible with the Redis protocol;

[0104] S2: When Access receives read and write command requests from the application client, parse the Key consistent hash value in the message and send it to the corresponding Redis shard node at the backend;

[0105] S3: Parse the message packet through the master instance. When receiving a write operation command from the client, mark the client and record the offset of the corresponding write command in the AOF file;

[0106] S4: And the master instance simultaneously appends the write operation command to the AOF file for disk writing;

[0107] S5: After the write operation command is written to the AOF file, the Redis master instance sends a corresponding AOF file transfer command to the peer Redis slave instance node;

[0108] S6: After the slave instance receives the AOF file transfer command from the master node, execute the command and sign for it;

[0109] S7: When the master instance receives the signed response command from the slave instance, update the confirmation offset of the slave node and reply to the event to respond to the client;

[0110] S8: When the network latency between the master and slave nodes is high, the Redis master-slave data synchronization progress has not caught up and the master-slave data synchronization progress has stalled, and the stall time is greater than a preset threshold, which is 1 minute, that is, mark the current master-slave node data as "out of sync" state;

[0111] S9: When the master-slave data synchronization status is "out of sync", automatically downgrade the master-slave replication mechanism from the semi-synchronous mechanism to the asynchronous replication mechanism. When the network between the master and slave nodes is restored, update the master-slave data status to "synchronized", and automatically restore the asynchronous replication mechanism to the semi-synchronous mechanism again.

[0112] Embodiment 5

[0113] I. Write and run the distributed cache proxy layer service Access

[0114] First, write and run the distributed cache proxy layer service Access using a preset programming language (such as Java). Access is responsible for receiving read and write command requests from the application client, parsing the Key consistent hash value in the packet, and sending it to the corresponding Redis shard node at the backend.

[0115] II. The master instance parses the message packet and marks the write operation command

[0116] When the master instance receives the write operation command from the client, it marks the client and records the offset of the corresponding write command in the AOF file. In this way, when the slave instance needs to synchronize data, it can obtain the write operation command from the AOF file.

[0117] III. The master instance appends the write operation command to the AOF file and flushes it to disk

[0118] The master instance simultaneously appends the write operation command to the AOF file and flushes it to disk. This operation depends on the current disk flushing policy configured for this node. After the write operation command is flushed to the AOF file, the Redis master instance sends the corresponding AOF file transfer command to the peer Redis slave instance node.

[0119] IV. The slave instance executes the command and acknowledges

[0120] After receiving the AOF file transfer command from the master node, the slave instance executes the command and acknowledges. At this time, the slave node sends an acknowledgment response command to the master node before flushing to disk. In this way, the master instance can know that the slave instance has successfully received and executed the write operation command.

[0121] V. Update the confirmation offset of the slave node and reply to the event to respond to the client

[0122] When the master instance receives the acknowledgment response command from the slave instance, it updates the confirmation offset of the slave node and replies to the event to respond to the client. In this way, the client can know that the write operation command has been successfully executed.

[0123] VI. Mark the master-slave node data as "out of sync" and downgrade the replication mechanism

[0124] When the network latency between the master and slave nodes is high, the progress of Redis master-slave data synchronization has not caught up and the progress of master-slave data synchronization has stalled, and the stall time is greater than a preset threshold (such as 1 minute), then the current master-slave node data is marked as "out of sync" status. At this time, the master-slave replication mechanism is automatically downgraded from the semi-synchronous mechanism to the asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data status is updated to "synchronized", and the asynchronous replication mechanism is automatically restored to the semi-synchronous mechanism again.

[0125] VII. Application Scenarios and Advantages

[0126] The method for semi-synchronous caching data provided by the embodiments of the present invention can be applied to various scenarios that require a distributed caching system, such as e-commerce systems, social applications, etc. By using the consistent hashing algorithm for cache proxy and the master-slave replication mechanism for data synchronization, the efficiency and reliability of the cache can be effectively improved. At the same time, by automatically detecting network latency and replication progress stagnation, the problem of data out of sync can be avoided, and the availability and stability of the system can be improved. In addition, the embodiments of the present invention can also be combined with other cache synchronization methods to improve data reliability and consistency.

[0127] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus.

[0128] The memory is used to store a computer program;

[0129] The processor is used to implement the method for semi-synchronous caching data of the present invention when executing the program stored in the memory.

[0130] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the above terminal and other devices. The memory can include a Random Access Memory (RAM), and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage system located far from the aforementioned processor.

[0131] The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0132] In addition, to achieve the above object, an embodiment of the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for semi-synchronous caching data in the embodiment of the present invention.

[0133] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the embodiments of the present invention can take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a system for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0135] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction system that implements the functions in Figure 1 one or more flows and / or blocksFigure 1 The functions specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable terminal device provide for implementing the steps of the functions specified in one or more processes and / or boxes. Figure 1 One process or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes.

[0137] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "And / or" means that either one of the two can be selected, or both can be selected. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusively, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or terminal device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or terminal device including the element.

[0138] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

[0139] In summary, the embodiment of the present invention provides a method for semi-synchronous caching data, which can effectively implement functions such as proxy and sharding of caching data, data synchronization and replication mechanism management, etc., improve the efficiency and reliability of the distributed caching system, and provide an effective solution for related applications.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for semi-synchronous caching data, characterized in that, It includes the following steps: S1: Write and run the distributed cache proxy layer service Access using a preset programming language; S2: When Access receives read and write command requests from the application client, parse the Key consistent hash value in the message and send it to the corresponding Redis shard node at the backend; S3: Parse the message packet through the master instance. When receiving a write operation command from the client, mark the client and record the offset of the corresponding write command in the AOF file; S4: And the master instance simultaneously appends the write operation command to the AOF file for disk write; S5: After the write operation command is written to the AOF file for disk, the Redis master instance sends a corresponding AOF file transfer command to the peer Redis slave instance node; S6: After the slave instance receives the AOF file transfer command from the master node, execute the command and sign for it; S7: When the master instance receives the signed response command from the slave instance, update the confirmation offset of the slave node and reply to the event to respond to the client; S8: When the network latency between the master and slave nodes is relatively high, the Redis master-slave data synchronization progress has not caught up and the master-slave data synchronization progress has stalled, and the stall time is greater than the preset threshold, then mark the current master-slave node data as the "out-of-sync" state; S9: When the master-slave data synchronization state is "out-of-sync", automatically downgrade the master-slave replication mechanism from the semi-synchronous mechanism to the asynchronous replication mechanism. When the network between the master and slave nodes is restored, the master-slave data state is updated to "synchronized", and automatically restore the asynchronous replication mechanism to the semi-synchronous mechanism again.

2. The method for semi-synchronous caching data according to claim 1, wherein In S8, the threshold is 1 minute.

3. A method for semi-synchronous caching data according to claim 1, characterized in that, The preset programming language is Java.

4. A method for semi-synchronous caching data according to claim 1, characterized in that, The distributed cache is a NoSQL in-memory database software.

5. A method for semi-synchronous caching data according to claim 4, characterized in that, The NoSQL in-memory database software is compatible with the Redis protocol.

6. A method for semi-synchronous caching data according to claim 1, characterized in that The marking includes establishing different data synchronization channels between the master and slave nodes.

7. A method for semi-synchronous caching data according to claim 1, characterized in that, The AOF file is a log file of commands.

8. A method for semi-synchronous caching of data according to claim 7, characterized in that: The AOF file also includes the timestamp and operation content information of the write operation command.

9. A method for semi-synchronous caching data according to claim 1, characterized in that, The downgrading also includes caching part of the data to the local hard disk.

10. A method for semi-synchronous caching data according to claim 8, characterized in that, The master instance also includes writing the current timestamp into the confirmation offset of the slave node.

Images