Data update methods, devices, and electronic equipment for Redis data caching

Abstract

This application discloses a data update method, apparatus, and electronic device for Redis data caching. Relating to the field of financial technology, the method includes: detecting whether the key values ​​corresponding to M key-value pairs to be verified are the same; when the key values ​​of any two target key-value pairs among the M target key-value pairs are different, determining that the data is inconsistent among the M storage nodes in the distributed storage system, and detecting whether the data update times corresponding to the M target key-value pairs are the same based on the target timestamp information; when the data update times corresponding to any two target key-value pairs are different, determining a first key-value pair based on the target timestamp information, and updating the key-value pair data corresponding to the storage node based on the first key-value pair data. This application solves the technical problem of low data update efficiency caused by the inability to automatically update data when there is data inconsistency among multiple storage nodes in a distributed storage system.

Description

Technical Field

[0001] This application relates to the field of financial technology, and more specifically, to a method, apparatus, and electronic device for updating data in a Redis data cache. Background Technology

[0002] In existing technologies, distributed caching multi-site active-active architecture achieves data synchronization between replicas across three campuses through a data synchronization component, thus realizing high availability across campuses and geographically dispersed locations, as well as eventual data consistency. Verification of data consistency across storage nodes in each campus is typically achieved by comparing the number of keys and the memory usage of the storage nodes across the three campuses. However, this verification method has the following drawbacks:

[0003] 1. Data consistency is verified by comparing the number of keys and memory usage in the storage nodes of each park. However, this method is too macroscopic and cannot accurately verify some special scenarios of data inconsistency, resulting in a low accuracy rate of data consistency verification.

[0004] 2. When the data between the storage nodes in different parks is inconsistent, the data cannot be updated automatically, resulting in low data update efficiency.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This application provides a data update method, apparatus, and electronic device for Redis data caching, to at least solve the technical problem of low data update efficiency caused by the inability to automatically update data when data is inconsistent among multiple storage nodes in a distributed storage system.

[0007] According to one aspect of this application, a data update method for Redis data cache is provided, comprising: retrieving M target key-value pairs to be verified and target timestamp information corresponding to each target key-value pair from a target hash table, wherein each target key-value pair is a key-value pair corresponding to a storage node in a distributed storage system, each target timestamp information is used to characterize the data update time of the target key-value pair corresponding to the target timestamp information, the primary keys of the M target key-value pairs are the same, and M is a positive integer; detecting whether the key values ​​corresponding to the M target key-value pairs are the same; and updating any one of the M target key-value pairs. If the key values ​​corresponding to two target key-value pairs are different, it is determined that the data is inconsistent among the M storage nodes in the distributed storage system, and the update times of the data corresponding to the M target key-value pairs are checked based on the timestamp information of each target. If the update times of any two target key-value pairs are different, the first key-value pair is determined based on the timestamp information of each target, and the key-value pair data corresponding to the storage nodes in the distributed storage system is updated based on the first key-value pair data. The first key-value pair data is the target key-value pair data with the latest update time among the M target key-value pairs data.

[0008] Furthermore, before retrieving the M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair from the target hash table, the data update method for Redis data cache also includes: obtaining the target metadata corresponding to each storage node in the distributed storage system to obtain M target metadata; obtaining multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair based on the M target metadata; and storing the multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair into a preset hash table to obtain the target hash table.

[0009] Furthermore, after detecting whether the key values ​​corresponding to the M target key-value pairs are the same, the data update method for the Redis data cache also includes: when the key values ​​corresponding to the M target key-value pairs are the same, determining that the data among the M storage nodes in the distributed storage system is consistent, and deleting the M target key-value pairs from the target hash table.

[0010] Furthermore, after detecting whether the data update times corresponding to the M target key-value pairs are the same based on each target timestamp information, the data update method for Redis data cache also includes: if the data update times corresponding to the M target key-value pairs are the same, dividing the M target key-value pairs into N data sets, wherein the key values ​​corresponding to the target key-value pairs in each data set are the same, M is greater than or equal to N, and N is a positive integer; when M is greater than N, selecting the data set with the largest number of target key-value pairs among the N data sets as the target data set; determining the target key-value pairs in the target data set as the second key-value pairs; and updating the key-value pairs corresponding to the storage nodes in the distributed storage system based on the second key-value pairs.

[0011] Furthermore, after dividing the M target key-value pairs into N data sets, the data update method for the Redis data cache also includes: when M equals N, obtaining the node index corresponding to each target key-value pair; obtaining a third key-value pair from the M target key-value pairs based on each node index, wherein the third key-value pair is the target key-value pair with the largest node index among the M target key-value pairs; and updating the key-value pair data corresponding to the storage nodes in the distributed storage system based on the third key-value pair data.

[0012] Furthermore, the process of obtaining the third key-value pair data from the M target key-value pair data based on each node index includes: sorting the M target key-value pair data based on each node index to obtain a sorting result; and obtaining the third key-value pair data from the M target key-value pair data based on the sorting result.

[0013] Furthermore, updating the key-value pair data corresponding to the storage node in the distributed storage system based on the first key-value pair data includes: responding to a data update instruction, wherein the data update instruction carries unique identification information corresponding to the storage node to be updated; sending the first key-value pair data to the storage node to be updated based on the unique identification information; and updating the key-value pair data corresponding to the storage node to be updated according to the first key-value pair data.

[0014] According to another aspect of this application, a data update device for Redis data caching is also provided, comprising: an acquisition module, configured to acquire M target key-value pairs to be verified and target timestamp information corresponding to each target key-value pair from a target hash table, wherein each target key-value pair is a key-value pair corresponding to a storage node in a distributed storage system, and each target timestamp information is used to characterize the data update time of the target key-value pair corresponding to the target timestamp information, the primary keys of the M target key-value pairs are the same, and M is a positive integer; a detection module, configured to detect whether the key values ​​corresponding to the M target key-value pairs are the same; and a determination module, configured to determine whether the M target key-value pairs are identical. If any two target key-value pairs in the data have different key values, determine that the data between the M storage nodes in the distributed storage system is inconsistent, and detect whether the data update times corresponding to the M target key-value pairs are the same based on the target timestamp information; the update module is used to determine the first key-value pair based on the target timestamp information when the data update times corresponding to any two target key-value pairs in the M target key-value pairs are different, and update the key-value pairs corresponding to the storage nodes in the distributed storage system based on the first key-value pair, wherein the first key-value pair is the target key-value pair with the latest data update time among the M target key-value pairs.

[0015] According to another aspect of this application, a computer-readable storage medium is also provided, in which a computer program is stored, wherein the computer program is configured to execute the above-described data update method for the Redis data cache at runtime.

[0016] According to another aspect of this application, an electronic device is also provided, the electronic device including one or more processors; a memory for storing one or more programs, which, when executed by one or more processors, cause one or more processors to implement the program for running the program, wherein the program is configured to execute the above-described data update method for Redis data cache during runtime.

[0017] In this application, firstly, M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair are obtained from the target hash table. Each target key-value pair is a key-value pair corresponding to a storage node in a distributed storage system. Each target timestamp information is used to characterize the data update time of the target key-value pair corresponding to that timestamp. The primary keys of the M target key-value pairs are the same, and M is a positive integer. Then, it is checked whether the keys corresponding to the M target key-value pairs are the same. Then, the keys corresponding to any two of the M target key-value pairs are checked. If the values ​​are different, it is determined that the data among the M storage nodes in the distributed storage system is inconsistent, and the update times of the data corresponding to the M target key-value pairs are checked based on the target timestamp information. Finally, if the update times of any two target key-value pairs are different, the first key-value pair is determined based on the target timestamp information, and the key-value pair data corresponding to the storage nodes in the distributed storage system is updated based on the first key-value pair data. The first key-value pair data is the target key-value pair data with the latest update time among the M target key-value pairs data.

[0018] In the above process, by detecting whether the key values ​​corresponding to M target key-value pairs are the same, the consistency of data among M storage nodes in the distributed storage system is determined. This approach is applicable to all data consistency verification scenarios, improving the accuracy of data consistency verification and effectively avoiding business logic errors caused by data inconsistency. When data inconsistency is determined among M storage nodes in the distributed storage system, the first key-value pair is determined based on the target timestamp information. Based on the first key-value pair, the corresponding key-value pair data of the storage nodes in the distributed storage system can be automatically updated, thereby improving the technical effect of data update efficiency. This solves the technical problem of low data update efficiency caused by the inability to automatically update data when there is data inconsistency among multiple storage nodes in the distributed storage system. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a flowchart of an optional data update method for Redis data cache according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of an optional data update process for Redis data cache according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram illustrating an optional target hash table generation process according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of an optional data update device for Redis data cache according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] It should be noted that the data update method, apparatus and electronic device for Redis data caching in this application can be used in the fintech field, as well as in other fields. The application fields of the data update method, apparatus and electronic device for Redis data caching in this application are not limited.

[0028] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0029] Example 1

[0030] According to an embodiment of this application, a method embodiment for updating data in a Redis data cache is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] Figure 1 This is a flowchart of an optional data update method for Redis data cache according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0032] To facilitate understanding of the technical solutions of this application, the following are specific definitions of some key terms used in this application:

[0033] Redis data cache is a key-value storage system that stores data types including string, list, set, sorted set (zset), and hash. Each data type has different data operation methods.

[0034] unixTime: The number of milliseconds that have elapsed since midnight on January 1, 1970, to the present moment.

[0035] Step S101: Obtain the M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair from the target hash table.

[0036] In one optional embodiment, a data update system for Redis data caching can serve as the execution entity for the data update method for Redis data caching in this application embodiment. For ease of description, the data update system for Redis data caching will be referred to as the system below.

[0037] In step S101, each target key-value pair is a key-value pair corresponding to a storage node in the distributed storage system. Each target timestamp is used to characterize the data update time of the target key-value pair corresponding to that timestamp. The primary keys of the M target key-value pairs are the same, and M is a positive integer. The target timestamp consists of unixTime and the node ID of the storage node.

[0038] For example, when the number of storage nodes M in a distributed storage system is 3, such as Figure 2As shown, the system can obtain a target key-value pair data corresponding to each storage node of the distributed storage system and the target timestamp information corresponding to each target key-value pair data through the target hash table, and obtain a total of 3 target key-value pair data. Among them, the 3 target key-value pair data can be [key (i.e., the primary key corresponding to the key-value pair) = 1, value (i.e., the key value corresponding to the key-value pair) = 2], [key = 1, value = 3], [key = 1, value = 3]; the 3 target key-value pair data can also be [key = 1, value = 3], [key = 1, value = 3], [key = 1, value = 3].

[0039] In this embodiment, to achieve automatic data updates when data inconsistencies exist among multiple storage nodes in a distributed storage system, this application has made secondary development modifications to the Redis data cache. Specifically, this application has modified the underlying data structure of the Redis kernel source code—redisObject—by adding hybrid clock metadata to determine the key update time. A specific code example is as follows:

[0040] typedef struct redisObject{

[0041] unsigned type:4;

[0042] unsigned encoding:4;

[0043] unsigned lru:LRU_BITS; / *LRU time(relative to global lru_clock)or

[0044] *LFU data(least significant 8bits frequency

[0045] *and most significant 16bits access time).* /

[0046] int refcount;

[0047] void*ptr;

[0048] / / Added hybrid clock attribute

[0049] HLC timestamp;

[0050] }robj;

[0051] typedef struct hybirdLogicClock{

[0052] unsigned long unixTime;

[0053] char* nodeId;

[0054] }hlc;

[0055] Step S102: Detect whether the key values ​​corresponding to the M key-value pairs are the same.

[0056] To improve the accuracy of data consistency verification and avoid business logic errors caused by data inconsistency, in this embodiment, as follows: Figure 2 As shown, the system can determine whether the data among the M storage nodes in the distributed storage system is consistent by detecting whether the key values ​​corresponding to the M target key-value pairs are the same.

[0057] Step S103: If any two target key-value pairs in the M target key-value pairs have different key values, determine that the data between the M storage nodes in the distributed storage system is inconsistent, and detect whether the data update times corresponding to the M target key-value pairs are the same based on the target timestamp information.

[0058] In this embodiment, as Figure 2 As shown, the system can determine data inconsistency among M storage nodes in a distributed storage system if any two of the target key-value pairs have different key values. For example, when the three target key-value pairs are [key=1, value=2], [key=1, value=3], and [key=1, value=3], or when the three target key-value pairs are [key=1, value=2], [key=1, value=3], and [key=1, value=4], the system determines that the data is inconsistent among the three storage nodes in the distributed storage system.

[0059] Optionally, such as Figure 2 As shown, in order to automatically update data when there is inconsistency among the M storage nodes in a distributed storage system, the system can detect whether the data update times corresponding to the M target key-value pairs are the same based on the target timestamp information, and then determine to update the data of the storage nodes in the distributed storage system.

[0060] Step S104: If the update times of any two target key-value pairs among the M target key-value pairs are different, determine the first key-value pair based on the timestamp information of each target, and update the key-value pair data corresponding to the storage node in the distributed storage system based on the first key-value pair data.

[0061] In step S104, the first key-value pair data is the target key-value pair data with the latest data update time among the M target key-value pair data.

[0062] In this embodiment, as Figure 2 As shown, when any two target key-value pairs among M target key-value pairs have different update times, the system determines the first key-value pair based on the timestamp information of each target. The system can select the target timestamp information corresponding to the latest data update time as the first key-value pair. For example, when there are 3 target key-value pairs and their corresponding update times are [key=1, value=2], 1649595836000 (corresponding to April 10, 2022, 21:03:56), [key=1, value=3], 1649595838000 (corresponding to April 10, 2022, 21:03:58), [key=1, value=4], 1649595839000 (corresponding to April 10, 2022, 21:03:59), the system can select [key=1, value=4] as the first key-value pair. The system can then send the first key-value pair data to the storage node to be updated, and update the corresponding key-value pair data of the storage node to be updated based on the first key-value pair data. This enables automatic data updates when data is inconsistent between multiple storage nodes in a distributed storage system, thereby improving data update efficiency.

[0063] Based on the scheme defined in steps S101 to S104 above, it can be understood that in this embodiment, firstly, M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair are obtained from the target hash table. Each target key-value pair is a key-value pair corresponding to a storage node in a distributed storage system. Each target timestamp information is used to characterize the data update time of the target key-value pair corresponding to that timestamp information. The primary keys of the M target key-value pairs are the same, and M is a positive integer. Then, it is detected whether the key values ​​corresponding to the M target key-value pairs are the same. Then, in the M target key-value pairs... If any two target key-value pairs have different key values, the data inconsistency among the M storage nodes in the distributed storage system is determined, and the update times of the data corresponding to the M target key-value pairs are checked based on the timestamp information of each target. Finally, if the update times of any two target key-value pairs among the M target key-value pairs are different, the first key-value pair is determined based on the timestamp information of each target, and the key-value pair data corresponding to the storage nodes in the distributed storage system is updated based on the first key-value pair data. The first key-value pair data is the target key-value pair data with the latest update time among the M target key-value pairs.

[0064] It should be noted that in the above process, determining whether the data between the M storage nodes in the distributed storage system is consistent by detecting whether the key values ​​corresponding to the M target key-value pairs are the same can be adapted to all data consistency verification scenarios, improving the accuracy of data consistency verification and effectively avoiding business logic errors caused by data inconsistency. When data inconsistency is determined among the M storage nodes in the distributed storage system, the first key-value pair is determined based on the target timestamp information, and the corresponding key-value pair data of the storage nodes in the distributed storage system can be automatically updated based on the first key-value pair data, thereby achieving the technical effect of improving data update efficiency. This solves the technical problem of low data update efficiency caused by the inability to automatically update data when there is data inconsistency among multiple storage nodes in the distributed storage system.

[0065] Optionally, in the data update method for Redis data caching provided in this application embodiment, before obtaining the M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair from the target hash table, the system can obtain the target metadata corresponding to each storage node in the distributed storage system to obtain M target metadata; then, based on the M target metadata, obtain the multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair; finally, the system stores the multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair into a preset hash table to obtain the target hash table.

[0066] In this embodiment, as Figure 2 As shown, the system can obtain the target metadata corresponding to each storage node in the distributed storage system, and then start... Figure 3 The M thread tasks shown are the same number of storage nodes as in the distributed storage system. Each thread task iterates through a target metadata set, obtaining multiple key-value pairs for each storage node and their corresponding timestamps. These key-value pairs and timestamps are then stored in a pre-defined hash table, resulting in the target hash table. Optionally, the values ​​corresponding to the same key in the target hash table are arrays, where the number of elements is the number of storage nodes, and each element is the value corresponding to the key in a storage node. For example, as shown... Figure 3 As shown, the distributed storage system has 6 storage nodes. The system uses a thread task to traverse the target metadata corresponding to a node, and obtains the value array corresponding to key1, including value1 and timestamp1 corresponding to storage node 1, value2 and timestamp2 (i.e., timestamp information) corresponding to storage node 2, value2 and timestamp2 corresponding to storage node 3, value2 and timestamp2 corresponding to storage node 4, value2 and timestamp2 corresponding to storage node 5, value1 and timestamp1 corresponding to storage node 6; and obtains the value array corresponding to key2, etc.

[0067] It should be noted that the system obtains multiple key-value pairs and timestamp information corresponding to each storage node based on M target metadata, and stores the multiple key-value pairs and timestamp information corresponding to each key-value pair in a preset hash table to obtain the target hash table, which prepares the data for subsequent retrieval of target key-value pairs and target timestamp information through the target hash table.

[0068] Optionally, in the data update method for Redis data cache provided in the embodiments of this application, after detecting whether the key values ​​corresponding to M key-value pairs are the same, the system can determine that the data between the M storage nodes in the distributed storage system is consistent when the key values ​​corresponding to the M target key-value pairs are the same, and delete the M target key-value pairs from the target hash table.

[0069] For example, such as Figure 2 As shown, when the three target key-value pairs are [key=1, value=3], [key=1, value=3], and [key=1, value=3] respectively, the system determines that the data among the three storage nodes in the distributed storage system is consistent, and then deletes the above three target key-value pairs from the target hash table.

[0070] It should be noted that by deleting the M target key-value pairs from the target hash table when data consistency is determined among the M storage nodes in the distributed storage system, the problem of wasting resources caused by updating data on storage nodes with consistent data can be avoided.

[0071] In one optional embodiment, after detecting whether the data update times corresponding to M target key-value pairs are the same based on each target timestamp information, the system can divide the M target key-value pairs into N data sets if the data update times corresponding to the M target key-value pairs are the same. In each data set, the target key-value pairs correspond to the same key value, M is greater than or equal to N, and N is a positive integer. When M is greater than N, the system can select the data set with the largest number of target key-value pairs among the N data sets as the target data set. Then, the system can determine the target key-value pairs in the target data set as the second key-value pairs. Finally, the system updates the key-value pairs corresponding to the storage nodes in the distributed storage system based on the second key-value pairs.

[0072] For example, such as Figure 2As shown, when the three target key-value pairs are [key=1, value=2], 1649595838000, [key=1, value=3], 1649595838000, [key=1, value=3], 1649595838000, the system determines that the data updates corresponding to the target key-value pairs are at the same time and that M is greater than N. Then, the system divides the three target key-value pairs into two sets: set 1 {[key=1, value=2]} and set 2 {[key=1, value=3], [key=1, value=3]}. The system determines that the number of target key-value pairs in set 1 is less than the number in set 2. Therefore, the system can use set 2 as the target set, and use key=1, value=3 in set 2 as the second key-value pairs, updating the corresponding key-value pairs on the storage nodes in the distributed storage system based on the second key-value pairs.

[0073] It should be noted that, in order to avoid the problem of not being able to automatically update the key-value pairs in the distributed storage system when the data corresponding to the M target key-value pairs are updated at the same time, due to the inability to determine the data to be updated in the storage node, the system divides the M target key-value pairs into N data sets. When M is greater than N, the target key-value pairs in the data set with the largest number of target key-value pairs are used as the second key-value pairs to update the corresponding key-value pairs in the distributed storage system. This achieves the technical effect of improving data update efficiency, and thus solves the technical problem of low data update efficiency caused by the inability to automatically update data when there is inconsistency among multiple storage nodes in the distributed storage system.

[0074] In another optional embodiment, after dividing the M target key-value pairs into N data sets, the system can obtain the node number corresponding to each target key-value pair when M equals N; then, based on each node number, it can obtain a third key-value pair from the M target key-value pairs, wherein the third key-value pair is the target key-value pair with the largest node number among the M target key-value pairs; finally, the system can update the key-value pair data corresponding to the storage nodes in the distributed storage system based on the third key-value pair data.

[0075] Optionally, obtaining the third key-value pair data from the M target key-value pair data based on each node number includes: the system can sort the M target key-value pair data based on each node number to obtain a sorting result; and then obtain the third key-value pair data from the M target key-value pair data based on the sorting result.

[0076] For example, such as Figure 2 As shown, when the three target key-value pairs are [key=1, value=2], 1649595838000, [key=1, value=3], 1649595838000, and [key=1, value=4], 1649595838000, the system determines that the data updates corresponding to the target key-value pairs are at the same time, and determines that M equals N. Then, the three target key-value pairs are divided into three data sets: data set 1 {[key=1, value=2]}, data set 2 {[key=1, value=3]}, and data set 23 {[key=1, value=4]}. The system then obtains the node index corresponding to each target key-value pair, including node index 001 for [key=1, value=2], node index 005 for [key=1, value=3], and node index 009 for [key=1, value=4]. The system then sorts the node numbers from largest to smallest, selects the first-ranked [key=1, value=4] as the third key-value pair, and updates the key-value pair data corresponding to the storage node in the distributed storage system based on [key=1, value=4].

[0077] It should be noted that, in order to avoid the problem of not being able to automatically update the key-value pairs in the distributed storage system when the data corresponding to the M target key-value pairs are updated at the same time and the M target key-value pairs are all different, the system determines the third key-value pair by the node sequence number corresponding to each target key-value pair, and updates the key-value pairs in the distributed storage system based on the third key-value pair. This achieves the technical effect of improving data update efficiency, and thus solves the technical problem of low data update efficiency caused by the inability to automatically update data when there is inconsistency among multiple storage nodes in the distributed storage system.

[0078] Optionally, in the data update method for Redis data caching provided in this application embodiment, updating the key-value pair data corresponding to the storage node in the distributed storage system based on the first key-value pair data includes: the system responding to a data update instruction, wherein the data update instruction carries unique identification information corresponding to the storage node to be updated; then the system can send the first key-value pair data to the storage node to be updated based on the unique identification information; finally, the system can update the key-value pair data corresponding to the storage node to be updated according to the first key-value pair data.

[0079] In this embodiment, the unique identifier information corresponding to the storage node can be the node ID of the storage node. For example, in response to a data update command, the system sends the first key-value pair data to storage node 1 based on the node ID of the storage node 1 to be updated carried in the data update command, and then the system updates the key-value pair data corresponding to storage node 1 based on the first key-value pair data.

[0080] It should be noted that, by responding to the data update command, the first key-value pair data is automatically sent to the storage node to be updated, and the key-value pair data corresponding to the storage node to be updated is updated according to the first key-value pair data. This achieves the technical effect of automatically updating the key-value pair data corresponding to the storage nodes in the distributed storage system, thereby improving the data update efficiency.

[0081] Therefore, the technical solution of this application can determine whether the data between M storage nodes in a distributed storage system is consistent by detecting whether the key values ​​corresponding to M target key-value pairs are the same. This can be adapted to all data consistency verification scenarios, improving the accuracy of data consistency verification and effectively avoiding business logic errors caused by data inconsistency. When data inconsistency is determined among M storage nodes in a distributed storage system, the key-value pairs corresponding to the storage nodes in the distributed storage system can be automatically updated, thereby improving the technical effect of data update efficiency. This solves the technical problem of low data update efficiency caused by the inability to automatically update data when data is inconsistent among multiple storage nodes in a distributed storage system.

[0082] Example 2

[0083] According to an embodiment of this application, an embodiment of a data update device for Redis data caching is provided, wherein... Figure 4 This is a schematic diagram of an optional data update device for Redis data cache according to an embodiment of this application, such as... Figure 4 As shown, the device includes:

[0084] The acquisition module 401 is used to acquire the M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair from the target hash table. Each target key-value pair is a key-value pair corresponding to a storage node in the distributed storage system. Each target timestamp information is used to characterize the data update time of the target key-value pair corresponding to the target timestamp information. The primary keys of the M target key-value pairs are the same, and M is a positive integer.

[0085] Detection module 402 is used to detect whether the key values ​​corresponding to M key-value pairs are the same;

[0086] The determination module 403 is used to determine the data inconsistency between the M storage nodes in the distributed storage system when the key values ​​corresponding to any two target key value pairs in the M target key value pairs are different, and to detect whether the data update times corresponding to the M target key value pairs are the same based on the timestamp information of each target.

[0087] The update module 404 is used to determine the first key-value pair based on the timestamp information of each target key-value pair when the data update times of any two target key-value pairs in the M target key-value pairs are different, and to update the key-value pairs corresponding to the storage nodes in the distributed storage system based on the first key-value pairs. The first key-value pairs are the target key-value pairs with the latest data update time among the M target key-value pairs.

[0088] It should be noted that the above-mentioned acquisition module 401, detection module 402, determination module 403 and update module 404 correspond to steps S101 to S104 in the above embodiment 1. The four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in the above embodiment 1.

[0089] Optionally, the data update device for Redis data caching further includes: a first acquisition module, used to acquire target metadata corresponding to each storage node in the distributed storage system before acquiring the M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair from the target hash table, thereby obtaining M target metadata; a second acquisition module, used to acquire multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair based on the M target metadata; and a storage module, used to store the multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair into a preset hash table, thereby obtaining a target hash table.

[0090] Optionally, the data update device for Redis data cache further includes: a first determining module, used to determine that the data among the M storage nodes in the distributed storage system is consistent when the key values ​​corresponding to the M key-value pairs are the same after detecting whether the key values ​​corresponding to the M target key-value pairs are the same, and to delete the M target key-value pairs from the target hash table.

[0091] Optionally, the data update device for Redis data caching further includes: a partitioning module, used to divide the M target key-value pairs into N data sets if the data update times corresponding to the M target key-value pairs are the same, after detecting whether the data update times corresponding to the M target key-value pairs are the same based on each target timestamp information, wherein the key values ​​corresponding to the target key-value pairs in each data set are the same, M is greater than or equal to N, and N is a positive integer; a second determining module, used to select the data set with the largest number of target key-value pairs in the N data sets as the target data set when M is greater than N; a third determining module, used to determine the target key-value pairs in the target data set as the second key-value pairs; and a first updating module, used to update the key-value pairs corresponding to the storage nodes in the distributed storage system based on the second key-value pairs.

[0092] Optionally, the data update device for Redis data caching further includes: a third acquisition module, used to acquire the node number corresponding to each target key-value pair when M equals N after dividing M target key-value pairs into N data sets; a fourth acquisition module, used to acquire a third key-value pair from the M target key-value pairs based on each node number, wherein the third key-value pair is the target key-value pair with the largest node number among the M target key-value pairs; and a second update module, used to update the key-value pair data corresponding to the storage nodes in the distributed storage system based on the third key-value pair data.

[0093] Optionally, the fourth acquisition module includes: a sorting unit, used to sort the M target key-value pairs based on the node number of each target key-value pair to obtain a sorting result; and an acquisition unit, used to acquire the third key-value pair data from the M target key-value pairs based on the sorting result.

[0094] Optionally, the update module includes: a response unit for responding to a data update instruction, wherein the data update instruction carries unique identifier information corresponding to the storage node to be updated; a sending unit for sending first key-value pair data to the storage node to be updated based on the unique identifier information; and an update unit for updating the key-value pair data corresponding to the storage node to be updated according to the first key-value pair data.

[0095] Example 3

[0096] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, in which a computer program is stored, wherein the computer program is configured to execute the above-described data update method for the Redis data cache at runtime.

[0097] Example 4

[0098] According to another aspect of the embodiments of this application, an electronic device is also provided, wherein, Figure 5 This is a schematic diagram of an optional electronic device according to an embodiment of this application, such as... Figure 5 As shown, the electronic device includes one or more processors; and a memory for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to run the programs, wherein the programs are configured to execute the aforementioned data update method for the Redis data cache at runtime.

[0099] like Figure 5 As shown, this application embodiment provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps:

[0100] Retrieve M target key-value pairs to be verified from the target hash table, along with the target timestamp information corresponding to each target key-value pair. Each target key-value pair represents a key-value pair corresponding to a storage node in the distributed storage system. Each target timestamp information represents the data update time of the target key-value pair corresponding to that timestamp. The M target key-value pairs have the same primary key, and M is a positive integer. Check whether the key values ​​corresponding to the M target key-value pairs are the same. If any two target key-value pairs have different key values... In the case of inconsistency among the M storage nodes in the distributed storage system, the update times of the M target key-value pairs are checked based on the timestamp information of each target. If the update times of any two target key-value pairs are different, the first key-value pair is determined based on the timestamp information of each target, and the key-value pairs corresponding to the storage nodes in the distributed storage system are updated based on the first key-value pair. The first key-value pair is the target key-value pair with the latest update time among the M target key-value pairs.

[0101] Optionally, the processor, when executing the program, further implements the following steps: before obtaining the M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair from the target hash table, obtains the target metadata corresponding to each storage node in the distributed storage system to obtain M target metadata; based on the M target metadata, obtains multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair; stores the multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair into a preset hash table to obtain the target hash table.

[0102] Optionally, the processor also performs the following steps when executing the program: after detecting whether the key values ​​corresponding to the M key-value pairs are the same, if the key values ​​corresponding to the M target key-value pairs are the same, it determines that the data between the M storage nodes in the distributed storage system is consistent, and deletes the M target key-value pairs from the target hash table.

[0103] Optionally, the processor further implements the following steps when executing the program: after detecting whether the data update times corresponding to the M target key-value pairs are the same based on each target timestamp information, if the data update times corresponding to the M target key-value pairs are the same, the M target key-value pairs are divided into N data sets, wherein the key values ​​corresponding to the target key-value pairs in each data set are the same, M is greater than or equal to N, and N is a positive integer; when M is greater than N, the data set with the largest number of target key-value pairs in the N data sets is selected as the target data set; the target key-value pairs in the target data set are determined as the second key-value pairs; and the key-value pairs corresponding to the storage nodes in the distributed storage system are updated based on the second key-value pairs.

[0104] Optionally, the processor, when executing the program, further implements the following steps: after dividing the M target key-value pairs into N data sets, when M equals N, obtains the node index corresponding to each target key-value pair; based on each node index, obtains a third key-value pair from the M target key-value pairs, wherein the third key-value pair is the target key-value pair with the largest node index among the M target key-value pairs; and updates the key-value pair data corresponding to the storage nodes in the distributed storage system based on the third key-value pair data.

[0105] Optionally, when the processor executes the program, it also performs the following steps: sorting the M target key-value pairs based on each node number to obtain a sorting result; and obtaining a third key-value pair from the M target key-value pairs based on the sorting result.

[0106] Optionally, when the processor executes the program, it further implements the following steps: responding to a data update instruction, wherein the data update instruction carries unique identification information corresponding to the storage node to be updated; sending first key-value pair data to the storage node to be updated based on the unique identification information; and updating the key-value pair data corresponding to the storage node to be updated according to the first key-value pair data.

[0107] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0108] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0112] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0113] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data update method for Redis data caching, characterized in that, include: Obtain the M target key-value pairs to be verified from the target hash table, as well as the target timestamp information corresponding to each target key-value pair. Each target key-value pair is a key-value pair corresponding to a storage node in the distributed storage system. Each target timestamp information is used to characterize the data update time of the target key-value pair corresponding to the target timestamp information. The primary keys of the M target key-value pairs are the same, and M is a positive integer. Detect whether the key values ​​corresponding to the M target key-value pairs are the same; If any two of the M target key-value pairs have different key values, it is determined that the data between the M storage nodes in the distributed storage system is inconsistent, and the update time of the data corresponding to the M target key-value pairs is detected based on the target timestamp information. If any two of the target key-value pairs have different data update times, a first key-value pair is determined based on each target timestamp information, and the key-value pair data corresponding to the storage node in the distributed storage system is updated based on the first key-value pair data. The first key-value pair data is the target key-value pair data with the latest data update time among the M target key-value pairs. Specifically, after detecting whether the data update times corresponding to the M target key-value pairs are the same based on each target timestamp information, if the data update times corresponding to the M target key-value pairs are the same, the M target key-value pairs are divided into N data sets, wherein the key values ​​corresponding to the target key-value pairs in each data set are the same, M is greater than or equal to N, and N is a positive integer; when M is greater than N, the data set with the largest number of target key-value pairs in the N data sets is selected as the target data set; the target key-value pairs in the target data set are determined as the second key-value pairs; and the key-value pairs corresponding to the storage nodes in the distributed storage system are updated based on the second key-value pairs. After dividing the M target key-value pairs into N data sets, when M equals N, the node number corresponding to each target key-value pair is obtained; based on each node number, a third key-value pair is obtained from the M target key-value pairs, wherein the third key-value pair is the target key-value pair with the largest node number among the M target key-value pairs; the key-value pairs corresponding to the storage nodes in the distributed storage system are updated based on the third key-value pair.

2. The method according to claim 1, characterized in that, Before retrieving the M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair from the target hash table, the method further includes: Obtain the target metadata corresponding to each storage node in the distributed storage system to obtain M target metadata items; Based on the M target metadata, obtain multiple key-value pairs of data corresponding to each storage node and the timestamp information corresponding to each key-value pair of data; The target hash table is obtained by storing multiple key-value pairs corresponding to each storage node and the timestamp information corresponding to each key-value pair into a preset hash table.

3. The method according to claim 1, characterized in that, After detecting whether the key values ​​corresponding to the M target key-value pairs are the same, the method further includes: If the key values ​​corresponding to the M target key-value pairs are the same, it is determined that the data among the M storage nodes in the distributed storage system is consistent, and the M target key-value pairs are deleted from the target hash table.

4. The method according to claim 1, characterized in that, Based on each node number, a third key-value pair is obtained from the M target key-value pairs, including: Sort the data based on the M target key values ​​according to each node number to obtain the sorting result; Based on the sorting result, the third key-value pair data is obtained from the M target key-value pair data.

5. The method according to claim 1, characterized in that, Updating the key-value pair data corresponding to the storage node in the distributed storage system based on the first key-value pair data includes: In response to a data update instruction, wherein the data update instruction carries unique identification information corresponding to the storage node to be updated; Based on the unique identifier information, the first key-value pair data is sent to the storage node to be updated; The key-value pair data corresponding to the storage node to be updated is updated based on the first key-value pair data.

6. A data update apparatus for Redis data cache, used to implement the data update method for Redis data cache as described in any one of claims 1 to 5, characterized in that, include: The acquisition module is used to acquire M target key-value pairs to be verified and the target timestamp information corresponding to each target key-value pair from the target hash table. Each target key-value pair is a key-value pair corresponding to a storage node in the distributed storage system. Each target timestamp information is used to characterize the data update time of the target key-value pair corresponding to the target timestamp information. The primary keys of the M target key-value pairs are the same, and M is a positive integer. The detection module is used to detect whether the key values ​​corresponding to the M target key-value pairs are the same; The determination module is used to determine that the data between the M storage nodes in the distributed storage system is inconsistent when any two of the target key-value pairs in the M target key-value pairs are different, and to detect whether the data update times corresponding to the M target key-value pairs are the same based on the target timestamp information of each target key-value pair. The update module is used to determine a first key-value pair based on each target timestamp when the update times of any two target key-value pairs among the M target key-value pairs are different, and to update the key-value pairs corresponding to the storage nodes in the distributed storage system based on the first key-value pairs, wherein the first key-value pairs are the target key-value pairs with the latest update time among the M target key-value pairs.

7. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program, wherein the computer program is configured to execute, at runtime, the data update method for the Redis data cache as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data update method for Redis data cache as described in any one of claims 1 to 5.

Images