A method, device and equipment for realizing bidirectional synchronization of cache suitable for reading and writing

Abstract

The present invention relates to the technical field of database bidirectional synchronization, and in particular to a cache bidirectional synchronization implementation method, device, equipment and storage medium suitable for reading and writing, including: obtaining NodeA request data, based on the NodeA request data, using a Lua script to perform update synchronization processing, obtaining request parameters, using the Lua script to assign the current time to a reqTime field, based on the reqTime field, judging whether the request parameter meets a preset value, obtaining new request parameters, based on the new request parameters, calling a service execution command, storing the new request data in a target node RedisA, if the target node RedisA changes, synchronizing the target node RedisA to a node RedisB, and based on the RedisShake native unidirectional synchronization mechanism, a mechanism for solving the Redis bidirectional synchronization loop problem in terms of architecture logic.

Description

Technical Field

[0001] The present invention relates to the technical field of database bidirectional synchronization, and in particular to a cache bidirectional synchronization implementation method, device, equipment and storage medium suitable for reading and writing. Background Art

[0002] Redis is a cache middleware that is frequently used in the industry. It is known for its high performance, high availability, high scalability, and rich ecology. Redis supports multiple deployment schemes such as stand-alone version, sentinel mode, and sharded cluster mode. Redis's underlying processing of business is single-threaded mode. Each request command will be queued inside Redis. There is a single thread inside Redis responsible for processing commands. By deploying Lua scripts on the Redis server, complex operations involving multiple reads and writes to Redis can be encapsulated into atomic commands that are executed at one time by the Redis service. Many business platforms in the industry with high concurrency and high availability requirements usually need to implement dual (multi) active disaster recovery, which often involves the synchronization of middleware data, such as cache data.

[0003] like Figure 1 As shown in the figure, the existing Redis Shake is a commonly used Redis synchronization tool in the industry, which supports synchronizing Redis data from one node to another. Figure 1 As shown: RedisShake startup is mainly divided into three steps. The first step is to execute the PSYNC command to start synchronization. The second step is to pull the RDB file from the source node to the target node. The third step is to synchronize the incremental command from the source node to the target node in real time. RedisShake is a mature Redis cross-node synchronization tool. The tool can well support one-way cache synchronization from node A to node B. However, as of the latest v3 version, because it does not have a database global TransationId mechanism, the native function cannot solve the loop problem in two-way synchronization well, so it cannot well support two-node two-way synchronization.

[0004] In summary, how to design a cache bidirectional synchronization implementation method suitable for reading and writing is a problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to provide a cache bidirectional synchronization implementation method suitable for reading and writing, so as to solve the problem that the native function of the existing RedisShake cannot well solve the loop problem in bidirectional synchronization and cannot well support the bidirectional synchronization of two nodes.

[0006] In order to solve the above technical problems, the present invention provides a cache bidirectional synchronization implementation method suitable for reading and writing, comprising:

[0007] Get NodeA request data;

[0008] Based on the NodeA request data, the Lua script is used to perform update synchronization processing to obtain request parameters;

[0009] Use the lua script to assign the current time to the reqTime field;

[0010] Based on the reqTime field, determine whether the request parameter meets the preset value, and obtain new request parameters;

[0011] Based on the new request parameters, call the service execution command and store the new request data in the target node RedisA;

[0012] If there is a change in the target node RedisA, the target node RedisA is synchronized to the node RedisB.

[0013] Preferably, the lua script is a Redis internal service script, which is responsible for encapsulating the logic of Redis cross-node read and write operations.

[0014] Preferably, judging whether the request parameter meets a preset value based on the reqTime field, and obtaining a new request parameter comprises:

[0015] Check whether the request parameter contains the value value. If so, and the time field in the value value is greater than the reqTime field, terminate the process; otherwise, regenerate the key value and value value to obtain a new request parameter.

[0016] Preferably, the value is encapsulated as structured data.

[0017] Preferably, it also includes:

[0018] Call the lua script where RedisA is located to query the key value;

[0019] Use the lua script to query the time values ​​of RedisA and RedisB respectively. If the current RedisA node has a record that is synchronized from the Redis B node and is newer than the Redis A node, the record of the RedisB node is returned this time, otherwise the record of the RedisA node is returned;

[0020] Call the RedisA service to execute commands.

[0021] Preferably, the disaster recovery node of RedisA is RedisB, and the disaster recovery node of RedisB is RedisA.

[0022] Preferably, if the target node RedisA changes, synchronizing the target node RedisA to the node RedisB includes:

[0023] If the target node RedisA changes, the cache synchronization middleware detects and obtains the incremental command and synchronizes it to the RedisB node.

[0024] The present invention also provides a cache bidirectional synchronization implementation device suitable for reading and writing, comprising:

[0025] Data acquisition module, obtains NodeA request data;

[0026] The data processing module uses the lua script to perform update synchronization based on the NodeA request data to obtain the request parameters;

[0027] The time assignment module uses the lua script to assign the current time to the reqTime field;

[0028] A judgment module, based on the reqTime field, judges whether the request parameter meets the preset value and obtains a new request parameter;

[0029] A service calling module calls a service execution command based on the new request parameters and stores the new request data in a target node RedisA;

[0030] The data synchronization module synchronizes the target node RedisA to the node RedisB if there is a change in the target node RedisA.

[0031] The present invention also provides a cache bidirectional synchronization implementation device suitable for reading and writing, comprising:

[0032] Memory for storing computer programs;

[0033] A processor is used to implement the steps of the above-mentioned method for implementing a bidirectional synchronization of a cache suitable for reading and writing when executing the computer program.

[0034] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned cache bidirectional synchronization implementation method suitable for reading and writing are implemented.

[0035] The present invention provides a cache bidirectional synchronization implementation method suitable for reading and writing. By configuring exclusive key prefix information for each node, the current node can only modify the key content of the current node prefix through the Redis of this node. Both Redis A and Redis B nodes can access the generated and modified data collection of Redis A and Redis B through local Redis. The mechanism solves the Redis bidirectional synchronization loop problem in terms of architectural logic. It is suitable for cache bidirectional synchronization of business platforms that rely on Redis read-write mode, and has high commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0037] Figure 1 This is a schematic diagram of the Redis Shake synchronization principle structure;

[0038] Figure 2 A flowchart of a first specific embodiment of a cache bidirectional synchronization implementation method suitable for reading and writing provided by the present invention;

[0039] Figure 3 This is a schematic diagram of the Redis update process structure;

[0040] Figure 4 To query the process logic diagram;

[0041] Figure 5 A structural block diagram of a cache bidirectional synchronization implementation device suitable for reading and writing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The core of the present invention is to provide a cache bidirectional synchronization implementation method, device, equipment and storage medium suitable for reading and writing. By configuring exclusive key prefix information for each node, Redis A or Redis B node can access the generated and modified data collection of Redis A and Redis B through local Redis, solving the Redis bidirectional synchronization loop problem in terms of architectural logic.

[0043] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] Please refer to Figure 1 , Figure 1This is a flowchart of a first specific embodiment of a cache bidirectional synchronization implementation method suitable for reading and writing provided by the present invention; the specific operation steps are as follows:

[0045] Step S101: Obtain NodeA request data;

[0046] Step S102: Based on the NodeA request data, update synchronization processing is performed using a Lua script to obtain request parameters;

[0047] The lua script is a Redis internal service script, which is responsible for encapsulating the logic of Redis cross-node read and write operations.

[0048] Step S103: using the lua script to assign the current time to the reqTime field;

[0049] Step S104: Based on the reqTime field, determine whether the request parameter meets the preset value, and obtain new request parameters;

[0050] Check whether the request parameter contains the value value. If so, and the time field in the value value is greater than the reqTime field, then end the process; otherwise, regenerate the key value and value value to obtain a new request parameter.

[0051] The value is encapsulated as structured data.

[0052] Step S105: Based on the new request parameters, call the service execution command and store the new request data in the target node RedisA;

[0053] Step S106: If the target node RedisA changes, synchronize the target node RedisA to the node RedisB;

[0054] If the target node RedisA changes, the cache synchronization middleware detects and obtains the incremental command and synchronizes it to the RedisB node.

[0055] Also includes:

[0056] Call the lua script where RedisA is located to query the key value;

[0057] Use the lua script to query the time values ​​of RedisA and RedisB respectively. If the current RedisA node has a record that is synchronized from the Redis B node and is newer than the Redis A node, the record of the RedisB node is returned this time, otherwise the record of the RedisA node is returned;

[0058] Call the RedisA service to execute commands.

[0059] The disaster recovery node of RedisA is RedisB, and the disaster recovery node of RedisB is RedisA.

[0060] This embodiment provides a cache bidirectional synchronization implementation method suitable for reading and writing, and is suitable for the implementation of cache bidirectional synchronization based on RedisShake tools in read and write mode. By configuring a dedicated node prefix for the key value generated by each node, bidirectional synchronization is implemented through innovation based on the native unidirectional synchronization mechanism of RedisShake tools. Lua scripts are deployed on the Redis server to be responsible for packaging and unpacking read and write operations. This solution is suitable for bidirectional synchronization in Redis read and write operation scenarios, has the advantages of zero intrusion to business applications, and solves the problem of loops in bidirectional synchronization.

[0061] Based on the above embodiment, this embodiment describes a cache bidirectional synchronization implementation method suitable for reading and writing, which is specifically as follows:

[0062] Suppose you need to implement bidirectional synchronization between Redis A and Redis B nodes, that is, the disaster recovery node of A is B, and the disaster recovery node of B is A. We introduce a mechanism similar to the global ID of the database in the Lua layer. The key value generated by each node is set with a prefix exclusive to the node. The current node can only modify the key generated by itself. Nodes A or B can access the data generated by both nodes A and B. Conventional key:value definition, in this proposal, the key value will be set to ${Pre}.${B}.${key}; the value is encapsulated as structured data, such as json format: {v:${value},t:${time}}, the time field represents the modification time of this record.

[0063] 1. Modify the cache logic, such as Figure 3 As shown;

[0064] NodeA requests the RedisA service Lua script (NodeA_Lua) to update data, and the request parameters are key and value;

[0065] NodeA_Lua assigns the reqTime field with the current time, and checks whether the field exists through ${Pre}.${A}.${key} and ${Pre}.${B}.${key}. If the field exists and the time field parsed from the value is greater than reqTime, the process ends and returns; otherwise, a new ${Pre}.${A}.${key} is generated as the key, and the value is {v:${value},t:${reqTime}}. Because the process is encapsulated in Lua scripts, it has the underlying execution atomicity.

[0066] NodeA_Lua calls the RedisA service to execute commands.

[0067] If RedisA changes, the cache synchronization middleware (Redis Shake) detects and obtains the incremental command and synchronizes it to the RedisB node;

[0068] 2. Read cache logic, such as Figure 4 As shown;

[0069] NodeA calls the Lua script (RedisA_Lua) where RedisA is located to query the key value.

[0070] RedisA_Lua uses ${Pre}.${A}.${KEY} and ${Pre}.${B}.${KEY} to query the results, read the latest record in time, unpack {v:${value},t:${time}} and return the value field. If the current Redis A node has a record that is synchronized from Redis B node and is newer than Redis A node, the record of Redis B node is returned this time, otherwise the record of Redis A node is returned.

[0071] RedisA_Lua calls the RedisA service to execute commands.

[0072] The present embodiment mentions a packaging and unpacking mechanism for implementing information such as the prefix and modification time of the key value in a Redis lua script, and the structuring is not limited to json, protobuf, xml and the like.

[0073] The embodiment of the present invention provides a cache bidirectional synchronization implementation method suitable for reading and writing, which is responsible for encapsulating the logic of Redis cross-node read and write operations by deploying and loading Lua scripts on the Redis server (Lua scripts can be considered as Redis internal service scripts). The A and B node business platforms and middleware servers such as Redis ensure that the time remains synchronized; for business applications, it is imperceptible and zero-intrusive; the Lua script is responsible for packaging and unpacking application requests and result responses, based on the RedisShake native one-way synchronization mechanism, by configuring exclusive key prefix information for each node, the current node can only modify the key content of the current node prefix through the Redis of this node, and both Redis A and Redis B nodes can access the generated and modified data collection of Redis A and Redis B through the local Redis, which is a mechanism to solve the problem of Redis bidirectional synchronization loop in terms of architecture logic.

[0074] Please refer to Figure 5 , Figure 5 A structural block diagram of a cache bidirectional synchronization implementation device suitable for reading and writing provided in an embodiment of the present invention; the specific device may include:

[0075] Data acquisition module 100, acquires NodeA request data;

[0076] The data processing module 200 performs update synchronization processing based on the NodeA request data using a Lua script to obtain request parameters;

[0077] The time assignment module 300 uses the lua script to assign the current time to the reqTime field;

[0078] The judgment module 400 judges whether the request parameter meets the preset value based on the reqTime field, and obtains a new request parameter;

[0079] The service calling module 500 calls the service execution command based on the new request parameters and stores the new request data in the target node RedisA;

[0080] The data synchronization module 600 synchronizes the target node RedisA to the node RedisB if there is a change in the target node RedisA.

[0081] A cache bidirectional synchronization implementation device suitable for reading and writing in this embodiment is used to implement the aforementioned cache bidirectional synchronization implementation method suitable for reading and writing. Therefore, the specific implementation method of a cache bidirectional synchronization implementation device suitable for reading and writing can be seen in the embodiment part of a cache bidirectional synchronization implementation method suitable for reading and writing in the previous text. For example, the data acquisition module 100, the data processing module 200, the time assignment module 300, the judgment module 400, the service call module 500, and the data synchronization module 600 are respectively used to implement steps S101, S102, S103, S104, S105, and S106 in the aforementioned cache bidirectional synchronization implementation method suitable for reading and writing. Therefore, its specific implementation method can refer to the description of the corresponding each part of the embodiment, which will not be repeated here.

[0082] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0083] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0084] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0085] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0086] The above is a detailed introduction to a cache bidirectional synchronization implementation method, device, equipment and storage medium suitable for reading and writing provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0087] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0088] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0089] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0090] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0091] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: In addition, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

Claims

1. A method for implementing bidirectional synchronization of a cache suitable for reading and writing, characterized in that: include: Get NodeA request data; Based on the NodeA request data, the Lua script is used to perform update synchronization processing to obtain request parameters, where the request parameters are key and value; Use the lua script to assign the current time to the reqTime field; Based on the reqTime field, determine whether the request parameter meets the preset value, and obtain new request parameters; Based on the new request parameters, call the service execution command and store the new request data in the target node RedisA; If the target node RedisA changes, synchronize the target node RedisA to node RedisB; Wherein, judging whether the request parameter meets the preset value based on the reqTime field, and obtaining the new request parameter includes: Check whether the request parameter contains the value value. If so, and the time field in the value value is greater than the reqTime field, terminate the process; otherwise, regenerate the key value and value value to obtain a new request parameter.

2. The method for implementing bidirectional synchronization of a cache suitable for reading and writing as claimed in claim 1, characterized in that: The lua script is a Redis internal service script, which is responsible for encapsulating the logic of Redis cross-node read and write operations.

3. The method for implementing bidirectional synchronization of cache suitable for reading and writing as claimed in claim 1, characterized in that: The value is encapsulated as structured data.

4. The method for implementing bidirectional synchronization of cache suitable for reading and writing as claimed in claim 1, characterized in that: Also includes: Call the lua script where RedisA is located to query the key value; Use the lua script to query the time values ​​of RedisA and RedisB respectively. If the current RedisA node has a record that is synchronized from the Redis B node and is newer than the Redis A node, the record of the RedisB node is returned this time, otherwise the record of the RedisA node is returned; Call the RedisA service to execute commands.

5. The method for implementing bidirectional synchronization of cache suitable for reading and writing as claimed in claim 4, characterized in that: The disaster recovery node of RedisA is RedisB, and the disaster recovery node of RedisB is RedisA.

6. The method for implementing bidirectional synchronization of cache suitable for reading and writing according to claim 1, characterized in that: If the target node RedisA changes, synchronizing the target node RedisA to the node RedisB includes: If the target node RedisA changes, the cache synchronization middleware detects and obtains the incremental command and synchronizes it to the RedisB node.

7. A cache bidirectional synchronization implementation device suitable for reading and writing, characterized in that: include: Data acquisition module, obtains NodeA request data; The data processing module uses the lua script to perform update synchronization based on the NodeA request data and obtains the request parameters, where the request parameters are key and value. The time assignment module uses the lua script to assign the current time to the reqTime field; A judgment module, based on the reqTime field, judges whether the request parameter meets the preset value and obtains a new request parameter; A service calling module calls a service execution command based on the new request parameters and stores the new request data in a target node RedisA; The data synchronization module synchronizes the target node RedisA to the node RedisB if there is a change in the target node RedisA; Wherein, the judgment module is also used for: Check whether the request parameter contains the value value. If so, and the time field in the value value is greater than the reqTime field, terminate the process; otherwise, regenerate the key value and value value to obtain a new request parameter.

8. A cache bidirectional synchronization implementation device suitable for reading and writing, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of a cache bidirectional synchronization implementation method suitable for reading and writing as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a cache bidirectional synchronization implementation method suitable for reading and writing as claimed in any one of claims 1 to 6 are implemented.

Images