Method, device and computer equipment for detecting data change between nodes

Abstract

The application relates to a method and device for detecting data change between nodes and a computer device, and relates to the technical field of computers. The method comprises the following steps: obtaining a first metadata difference record stored by a local node; the first metadata difference record is used for recording metadata information of difference metadata between the local node and a peer node; the metadata information at least comprises a metadata identifier; determining a data change detection result of the peer node according to a peer metadata change attribute of the first metadata difference record; the peer metadata change attribute is used for representing whether there is difference metadata of the corresponding metadata information in the peer node, which is not synchronized to the difference metadata record of the local node; the peer node is used for notifying the local node to update the peer metadata change attribute of the first metadata difference record in the case that data change occurs in the peer node. The method can reduce the required computing resources for perceiving data change between nodes.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for detecting data changes between nodes. Background Technology

[0002] In data backup systems, multi-node collaborative backup storage has become the mainstream architecture for ensuring data security. Therefore, each node in a data backup system needs to be aware of data changes between nodes.

[0003] In related technologies, data changes between nodes are usually detected by calculating and comparing fingerprints of data or data metadata; however, fingerprint calculation and comparison require a lot of computing resources, which leads to high computing resources required to detect data changes between nodes. Summary of the Invention

[0004] Therefore, it is necessary to address the technical problem of high computational resources required for data changes between sensing nodes by providing a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting data changes between nodes that can reduce the computational resources required for data changes between sensing nodes.

[0005] Firstly, this application provides a method for detecting data changes between nodes, including:

[0006] A method for detecting data changes between nodes, applied to a local node among multiple nodes, wherein the local node is any one of the multiple nodes; the method includes:

[0007] Obtain the first metadata difference record stored by the local node; the first metadata difference record is used to record metadata information of the difference metadata between the local node and the peer node; the peer node is any node among the plurality of nodes other than the local node; the metadata information includes at least the metadata identifier of the corresponding metadata;

[0008] Based on the record attribute information of the first metadata difference record, the data change detection result of the peer node is determined; the record attribute information of each metadata difference record includes at least a peer metadata change attribute; the peer metadata change attribute is used to characterize whether the corresponding peer node has different metadata information that has not been synchronized to the corresponding local node's metadata difference record; the peer node is used to notify the local node to update the peer metadata change attribute of the first metadata difference record when the peer node undergoes a data change.

[0009] Secondly, this application also provides a device for detecting data changes between nodes, including:

[0010] The difference record acquisition module is used to acquire the first metadata difference record stored by the local node; the local node is any one of a plurality of nodes; the first metadata difference record is used to record the metadata information of the difference metadata between the local node and the peer node; the peer node is any one of the plurality of nodes other than the local node; the metadata information includes at least the metadata identifier of the corresponding metadata.

[0011] The data change detection module is used to determine the data change detection result of the peer node based on the record attribute information of the first metadata difference record; the record attribute information of each metadata difference record includes at least a peer metadata change attribute; the peer metadata change attribute is used to characterize whether the corresponding peer node has different metadata information that has not been synchronized to the corresponding local node's metadata difference record; the peer node is used to notify the local node to update the peer metadata change attribute of the first metadata difference record when the peer node undergoes a data change.

[0012] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0013] Obtain the first metadata difference record stored on the local node; the local node is any one of multiple nodes; the first metadata difference record is used to record metadata information of the difference metadata between the local node and the peer node; the peer node is any one of the multiple nodes other than the local node; the metadata information includes at least the metadata identifier of the corresponding metadata;

[0014] Based on the record attribute information of the first metadata difference record, the data change detection result of the peer node is determined; the record attribute information includes at least the peer metadata change attribute; the peer metadata change attribute is used to characterize whether the peer node has corresponding metadata information that has not been synchronized to the local node's metadata difference record; when the local node is the local node, the peer node is the peer node, and when the peer node is the local node, the peer node is the local node.

[0015] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0016] Obtain the first metadata difference record stored on the local node; the local node is any one of a plurality of nodes; the first metadata difference record is used to record metadata information of the difference metadata between the local node and the peer node; the peer node is any one of the plurality of nodes other than the local node; the metadata information includes at least the metadata identifier of the corresponding metadata;

[0017] Based on the record attribute information of the first metadata difference record, the data change detection result of the peer node is determined; the record attribute information of each metadata difference record includes at least a peer metadata change attribute; the peer metadata change attribute is used to characterize whether the corresponding peer node has different metadata information that has not been synchronized to the corresponding local node's metadata difference record; the peer node is used to notify the local node to update the peer metadata change attribute of the first metadata difference record when the peer node undergoes a data change.

[0018] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0019] Obtain the first metadata difference record stored on the local node; the local node is any one of a plurality of nodes; the first metadata difference record is used to record metadata information of the difference metadata between the local node and the peer node; the peer node is any one of the plurality of nodes other than the local node; the metadata information includes at least the metadata identifier of the corresponding metadata;

[0020] Based on the record attribute information of the first metadata difference record, the data change detection result of the peer node is determined; the record attribute information of each metadata difference record includes at least a peer metadata change attribute; the peer metadata change attribute is used to characterize whether the corresponding peer node has different metadata information that has not been synchronized to the corresponding local node's metadata difference record; the peer node is used to notify the local node to update the peer metadata change attribute of the first metadata difference record when the peer node undergoes a data change.

[0021] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting data changes between nodes allow the local node to record metadata information, such as metadata identifiers, of the difference metadata between itself and the peer node based on its stored first metadata difference record. Based on the peer metadata change attributes of the first metadata difference record, the local node can determine whether the peer node has corresponding metadata information that has not been synchronized to its first metadata difference record. If the peer node has corresponding metadata information that has not been synchronized to its first metadata difference record, it indicates that the peer node has undergone a data change that has not been synchronized to its local node. In other words, the local node can quickly determine whether the peer node has undergone a data change and obtain the peer node's data change detection result through the peer metadata change attributes of the first metadata difference record. Based on the above process, each node can quickly determine whether other nodes have undergone data changes through the peer metadata change attributes of its stored metadata difference record. This process eliminates the need for fingerprint calculation and comparison, thus reducing the computational resources required to detect data changes between nodes. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a method for detecting data changes between nodes in one embodiment;

[0024] Figure 2 This is a flowchart illustrating the steps for obtaining the first metadata difference record in one embodiment;

[0025] Figure 3 This is a flowchart illustrating the steps of notifying the peer node that the local node has undergone data changes in one embodiment.

[0026] Figure 4 This is a flowchart illustrating the steps of sending a metadata change notification to a peer node in one embodiment.

[0027] Figure 5 This is a flowchart illustrating the steps of periodically sending change logs to the peer node in one embodiment;

[0028] Figure 6This is a flowchart illustrating the steps of negotiating grouping between the local node and the peer node in one embodiment;

[0029] Figure 7 This is a flowchart illustrating the steps of a local node determining the data change detection result of the peer node and synchronizing metadata difference records based on the data change detection result when the local node needs to use the data or metadata stored locally.

[0030] Figure 8 This is a flowchart illustrating the steps of updating the first metadata difference record and the corresponding record attribute information when the local node sends data changes in one embodiment.

[0031] Figure 9 This is a flowchart illustrating the steps of a local node periodically pushing change logs to a peer node in one embodiment.

[0032] Figure 10 This is a structural block diagram of a device for detecting data changes between nodes in one embodiment;

[0033] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0036] In one exemplary embodiment, such as Figure 1As shown, a method for detecting data changes between nodes is provided. In this embodiment, the method is applied to any one of the nodes in a multi-node collaborative data backup system. It is understood that each node can be a server, a terminal, or a system including both a server and a terminal. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. In this embodiment, the method includes the following steps S102 to S104:

[0037] Step S102: Obtain the first metadata difference record stored on the local node.

[0038] Each node stores at least one piece of data, and each piece of data has corresponding metadata.

[0039] Each node also stores metadata difference records for the other nodes. Each metadata difference record corresponds to two nodes and is used to record the metadata information of the difference metadata between the two corresponding nodes.

[0040] Each metadata entry includes at least its metadata identifier.

[0041] Specifically, the difference metadata between two nodes refers to the different metadata stored on the two nodes. This difference metadata includes metadata stored on the first node but not on the second node, and metadata stored on the second node but not on the first node. The first node can be either of the two nodes, and the second node is the node other than the first node. In practical applications, each metadata information recorded in the metadata difference record has a corresponding node identifier, used to identify the node among the two nodes corresponding to that metadata difference record that stores the difference metadata to which that metadata information belongs.

[0042] In this context, each node is referred to as the local node, and any node other than the local node among multiple nodes is called the peer node. The embodiments of this application are described using the local node as an example; it is easy to understand that the implementation process for the peer node is similar to that for the local node.

[0043] The first metadata difference record is a metadata difference record stored by the local node, used to record metadata information about the differences between the local node and the peer node. In specific applications, there is at least one peer node corresponding to the local node, and the local node stores the first metadata difference record corresponding to each peer node. The embodiments of this application use data change detection on one peer node as an example for illustration. It is easy to understand that the process of data change detection on multiple peer nodes is similar to the process of data change detection on one peer node.

[0044] Specifically, when a local node needs to use the data or metadata stored on its own end, it needs to detect whether the data on the peer node has changed. Therefore, the local node retrieves the first metadata difference record corresponding to the peer node from its own first database.

[0045] Step S104: Determine the data change detection result of the peer node based on the record attribute information of the first metadata difference record.

[0046] The record attribute information of each metadata difference record includes at least the peer metadata change attribute; the peer metadata change attribute of each metadata difference record is used to characterize whether there is any difference metadata in the peer node corresponding to the metadata difference record that has not been synchronized to the local node corresponding to the metadata difference record.

[0047] Among them, the peer node is used to notify the local node to update the peer metadata change attribute of the first metadata difference record when the peer node changes its data.

[0048] Specifically, the local node checks the peer metadata change attribute of the first metadata difference record to determine whether the peer node has corresponding metadata information that has not been synchronized to the first metadata difference record. If it does, it means that the peer node has changed data; if it does not, it means that the peer node has not changed data, and thus obtains the data change detection result of the peer node.

[0049] In the aforementioned method for detecting data changes between nodes, the local node, based on its stored first metadata difference record, can record metadata information of the difference metadata between itself and the peer node, such as the metadata identifier of the difference metadata. Based on the peer metadata change attribute of the first metadata difference record, the local node can determine whether the peer node has corresponding metadata information that has not been synchronized to its first metadata difference record. If the peer node has corresponding metadata information that has not been synchronized to its first metadata difference record, it indicates that the peer node has undergone a data change that has not been synchronized to the local node. In other words, the local node can quickly determine whether the peer node has undergone a data change through the peer metadata change attribute of the first metadata difference record, thus obtaining the data change detection result of the peer node. Based on this process, each node can quickly determine whether other nodes have undergone data changes through the peer metadata change attribute of its stored metadata difference record. In this process, it is no longer necessary to calculate or compare fingerprints, thereby reducing the computational resources required to perceive data changes between nodes.

[0050] In one exemplary embodiment, such as Figure 2 As shown, the first metadata difference record is obtained through the following steps:

[0051] Step S202: The local node sends a feature value range acquisition request to the peer node.

[0052] The feature value range acquisition request carries a target metadata feature, which is any one of multiple metadata features. The feature value of the metadata under the target metadata feature is the target feature value. In specific applications, the target metadata feature is the time information of the metadata, and the target feature value can be the generation time of the metadata or the latest modification time of the metadata.

[0053] Specifically, the local node arbitrarily selects one of multiple metadata features as the target metadata feature and sends a request to the peer node to obtain the feature value range of the target metadata feature.

[0054] In practical applications, metadata features with continuous characteristic values ​​are preferred as target metadata features.

[0055] In step S204, the peer node, in response to the feature value range acquisition request, returns the second feature value range to the local node.

[0056] Among them, the interval formed by the target feature values ​​of each second metadata stored by the peer node under the target metadata features is the second feature value interval.

[0057] Specifically, in response to the feature value range retrieval request, the peer node retrieves the second metadata stored by the peer node from the peer node's second database, determines the second feature value range based on the target feature value of each second metadata, and returns it to the local node.

[0058] Step S206: The local node obtains the global feature value range of the target metadata feature based on the union of the first feature value range and the second feature value range.

[0059] Among them, the interval formed by the target feature values ​​of each first metadata stored in the local node under the target metadata features is the first feature value interval.

[0060] Specifically, the local node retrieves the first metadata stored in the local node's first database, determines the first feature value interval based on the target feature value of each first metadata, and then takes the union of the first feature value interval and the second feature value interval to obtain the global feature value interval of the target metadata feature.

[0061] Step S208: The local node divides the global feature value interval into multiple first sub-intervals. Based on the multiple first sub-intervals and the target feature values ​​of each first metadata under the target metadata features, each first metadata is divided into multiple first metadata information sets.

[0062] Each first metadata information set corresponds to a first sub-interval, and the target feature value of each metadata information in each first metadata information set falls within the first sub-interval corresponding to that first metadata information set.

[0063] Specifically, the local node divides the global feature value range into multiple first sub-ranges, and divides the metadata information of each first metadata into multiple first metadata information sets based on the relationship between the target feature value of each first metadata and each first sub-range (whether it falls into the first sub-range).

[0064] In practical applications, if the target metadata feature is time information, the length of the first sub-interval is determined based on the data growth rate of the local node.

[0065] Step S210: The local node generates multiple metadata comparison requests based on multiple first sub-intervals and multiple first metadata information sets, and sends each metadata comparison request to the peer node.

[0066] Each metadata comparison request carries a first sub-interval and the metadata information in the first metadata information set corresponding to the first sub-interval.

[0067] Specifically, for each first metadata information set, the local node generates a metadata comparison request corresponding to the first metadata information set based on the first sub-interval corresponding to the metadata set and each metadata information in the first metadata information set; then, the local node sends each metadata comparison request to the peer node.

[0068] In practical applications, the local node can send each metadata comparison request sequentially or in parallel.

[0069] Step S212: In response to each metadata comparison request, the peer node compares the metadata identifier of the second metadata that falls into the first sub-interval carried in the metadata comparison request with the metadata identifiers carried in the metadata comparison request to obtain the difference metadata between the local node and the peer node under the metadata comparison request, stores the metadata information of the difference metadata under each metadata comparison request, obtains the second metadata difference record, and returns the metadata information of the difference metadata under each metadata comparison request to the local node.

[0070] Each metadata entry includes at least its metadata identifier; therefore, a metadata comparison request carrying metadata information carries the metadata identifier.

[0071] Specifically, for each metadata comparison request, the peer node retrieves the metadata identifier of the second metadata whose corresponding target feature value falls within the first sub-interval carried in the metadata comparison request from the peer node's second database; and compares the metadata identifiers carried in the metadata comparison request with the metadata identifiers of the second metadata retrieved from the second database to obtain the difference metadata between the local node and the peer node under the metadata comparison request; then, the peer node stores the metadata information of the difference metadata under each metadata comparison request in the peer node's second database to obtain the second metadata difference record stored by the peer node, and at the same time, the peer node returns the metadata information of the difference metadata under each metadata comparison request to the local node.

[0072] Step S214: The local node receives and stores the metadata information of the difference metadata under each metadata comparison request, and obtains the first metadata difference record.

[0073] Specifically, the local node receives the metadata information of the difference metadata under each metadata comparison request returned by the peer node, and stores the metadata information of the difference metadata under each metadata comparison request in the local node's first database, thus obtaining the first metadata difference record stored by the local node.

[0074] In this embodiment, on the one hand, nodes can quickly determine the difference metadata between their own nodes and peer nodes by comparing metadata identifiers, thereby obtaining their respective stored metadata difference records; on the other hand, nodes can negotiate target metadata features and global feature value ranges, and group the metadata information of each first metadata into multiple first metadata information sets based on the target feature value under the target metadata features. By generating corresponding metadata comparison requests for each first metadata information set and the corresponding first sub-range, nodes can perform metadata comparison on a unit of each first sub-range, thereby reducing the computing resources consumed by the comparison; furthermore, nodes can achieve metadata comparison through metadata identifiers, further reducing the computing resources consumed by the comparison.

[0075] In one exemplary embodiment, the metadata information for each metadata element also includes the target feature value of that metadata element.

[0076] In step 214 above, the local node receives and stores the metadata information of the difference metadata under each metadata comparison request to obtain the first metadata difference record. Specifically, this includes the following steps: dividing the global feature value interval into multiple second sub-intervals; based on the target feature values ​​in the metadata information of each difference metadata and the multiple second sub-intervals, dividing the metadata information of each difference metadata into multiple second metadata information sets; and determining the first metadata difference record corresponding to each second metadata information set.

[0077] Each second metadata information set corresponds to a second sub-interval, and the target feature value of the differential metadata to which each metadata information in each second metadata information set belongs falls within the second sub-interval corresponding to that second metadata information set.

[0078] Specifically, after receiving the metadata information of the difference metadata under the metadata comparison requests returned by the peer node, the local node first divides the global feature value interval again to obtain multiple second sub-intervals; then, based on the relationship between the target feature value of each difference metadata and each second sub-interval (whether it falls into the second sub-interval), the metadata information of each difference metadata is divided into multiple second metadata information sets; next, the local node stores each second metadata information set in the local node's first database to obtain each first metadata difference record stored by the local node.

[0079] In practical applications, step S212 above, where the peer node stores the metadata information of the difference metadata under each metadata comparison request to obtain the second metadata difference record, specifically includes the following steps: dividing the global feature value interval into multiple second sub-intervals; based on the target feature values ​​in the metadata information of each difference metadata and the multiple second sub-intervals, dividing the metadata information of each difference metadata into multiple second metadata information sets; and determining the second metadata difference record corresponding to each second metadata information set. That is, when constructing the second metadata difference record, the peer node also needs to divide the metadata information of each difference metadata into multiple second metadata information sets and obtain a second metadata difference record for each second metadata information set.

[0080] In practical applications, the local node and the peer node can negotiate the length of the second sub-interval.

[0081] In practical applications, the length of the second sub-interval can be the same as the length of the first sub-interval. If the length of the second sub-interval is the same as the length of the first sub-interval, since the local node has already divided the data into multiple first sub-intervals when generating the metadata comparison request, the local node does not need to divide the global feature value interval again.

[0082] In this embodiment, the node manages the metadata information of the compared difference metadata by dividing it into multiple second metadata information sets and determining the corresponding metadata difference records. This enables the group management of difference metadata, thereby reducing the amount of data query when updating metadata difference records and reducing the impact of local data changes on the global data.

[0083] In an exemplary embodiment, the record attribute information of each metadata difference record also includes the peer metadata change notification time; the peer metadata change notification time of each metadata difference record is the time when the peer node corresponding to the metadata difference record last sent a metadata change notification to the local node corresponding to the metadata difference record.

[0084] In step S104 above, after determining the data change detection result of the peer node based on the record attribute information of the first metadata difference record, the following steps are further included: if the peer metadata change attribute of the first metadata difference record is a first attribute value, then the data change detection result of the peer node is determined to be that the data has not changed; if the peer metadata change attribute of the first metadata difference record is a second attribute value, then the data change detection result of the peer node is determined to be that the data has changed; in the case that the data change detection result of the peer node is that the data has changed, obtain the first target change log corresponding to the log generation time that is later than or equal to the peer metadata change notification time of the first metadata difference record from each change log stored by the peer node; update the first metadata difference record based on the first target change log.

[0085] In practical applications, data changes include at least the addition and deletion of data; when data changes, there will be corresponding additions or deletions of metadata, and corresponding change logs will also be generated.

[0086] Specifically, if the peer metadata change attribute of the first metadata difference record is the first attribute value, such as 1 or true, then the local node determines that the data change detection result of the peer node is that the data has not changed, and therefore directly uses the data or metadata stored on the local node.

[0087] If the peer metadata change attribute of the first metadata difference record is the second attribute value, such as 0 or false, then the local node determines that the peer node's data change detection result is a data change. Therefore, it is necessary to first synchronize the peer node's data change status to the local node. That is, the local node retrieves the corresponding change logs whose log generation time is later than or equal to the peer metadata change notification time of the first metadata difference record from the change logs stored by the peer node, and uses them as the first target change logs. Then, based on the metadata information of the changed data in the first target change logs, the local node updates the first metadata difference record. Here, changed data refers to data that has undergone data changes.

[0088] It should be noted that updating the first metadata difference record on this node refers to adding or deleting metadata information of the changed data in the first metadata difference record, and does not necessarily involve adding or deleting the changed data and corresponding metadata in the first database of this node.

[0089] In practical applications, since the node manages the metadata information of the difference metadata in groups according to each second sub-interval, that is, the metadata information of each difference metadata is divided into metadata information (second metadata information set) under each second sub-interval, and each second sub-interval (second metadata information set) corresponds to a metadata difference information record, the node can also manage each change log according to each second sub-interval; for example, the node divides each change log into multiple change log sets according to the relationship between the target feature value of the metadata corresponding to each change log and each second sub-interval; where each change log set corresponds to a second sub-interval, and the target feature value of the metadata corresponding to each change log in each change log set falls into that change log and the corresponding second sub-interval.

[0090] In practical applications, since the nodes manage the metadata information of the difference metadata in groups according to each second sub-interval, the local node traverses each first metadata difference information record. If the peer metadata change attribute of the traversed first metadata difference information record is the second attribute value, the local node determines that the target feature value of the changed data of the peer node falls into the second sub-interval corresponding to the traversed first metadata difference information record. The local node determines the second sub-interval into which it falls as the first target sub-interval, and generates a change log retrieval request based on the first target sub-interval and the peer metadata change notification time of the first metadata difference record. The request is sent to the peer node to retrieve the change log with a log generation time later than or equal to the peer metadata change notification time of the first metadata difference record from the log change set stored by the peer node for the first target sub-interval.

[0091] In this embodiment, the local node can quickly determine whether the peer node has changed data by using the peer metadata change attribute of the first metadata difference record. By using the peer metadata change notification time of the first metadata difference record, the local node can actively pull the corresponding change log from the peer node to update the first metadata difference record.

[0092] In an exemplary embodiment, the record attribute information of each metadata difference record also includes the local metadata change attribute, the peer metadata change notification time, and the local metadata change notification time.

[0093] The local metadata change attribute of each metadata difference record is used to characterize whether the local node corresponding to the metadata difference record has corresponding metadata information that has not been synchronized to the metadata difference record of the peer node corresponding to the metadata difference record.

[0094] The peer metadata change notification time for each metadata difference record is the time when the peer node corresponding to that metadata difference record last sent a metadata change notification to the local node corresponding to that metadata difference record.

[0095] The local metadata change notification time for each metadata difference record is the time when the local node corresponding to the metadata difference record last sent a metadata change notification to the peer node corresponding to the metadata difference record.

[0096] Both the local metadata change attribute and the remote metadata change attribute are initialized to the first attribute value;

[0097] like Figure 3 As shown, the inter-node data change detection method provided in this application further includes the following steps to notify the peer node that a data change has occurred on the local node:

[0098] Step S302: If the local node detects a data change, it determines the data that has changed as the changed data, updates the first metadata difference record based on the metadata information of the changed data, and updates the local metadata change attribute of the first metadata difference record to the second attribute value.

[0099] Specifically, if the local node detects that its own data has changed, it will identify the changed data as the changed data and update the first metadata difference record based on the metadata information of the changed data. At the same time, the local metadata change attribute of the first metadata difference record will be updated to the second attribute value, for example, to 0 or false.

[0100] In practical applications, since the nodes manage the metadata information of the difference metadata in groups according to each second sub-interval, after determining the changed data, the local node also needs to determine the second target sub-interval in which the target feature value in the metadata information corresponding to the changed data falls within multiple second sub-intervals, based on the target feature value in the metadata information corresponding to the changed data. Then, based on the metadata information corresponding to the changed data, the first metadata difference information record and the corresponding local metadata change attribute corresponding to the second target sub-interval are updated.

[0101] In practical applications, when data changes occur on the local node, the first metadata difference record is updated according to Table 1:

[0102] Table 1

[0103]

[0104] Step S304: The local node sends a metadata change notification to the peer node and updates the local metadata change notification time in the first metadata difference record.

[0105] Specifically, the local node also needs to send a metadata change notification to the peer node and update the local metadata change notification time in the first metadata difference record to the generation time of the metadata change notification.

[0106] In practical applications, the local node updates the local metadata change notification time of the first metadata difference information record corresponding to the second target sub-interval of the changed data.

[0107] Step S306: In response to the metadata change notification, the peer node updates the peer metadata change attribute of the second metadata difference record stored by the peer node to the second attribute value, and updates the peer metadata change notification time of the second metadata difference record.

[0108] The second metadata difference record is used to record the difference metadata between the local node and the peer node.

[0109] Specifically, in response to the metadata change notification sent by the local node, the peer node updates the peer metadata change attribute of the second metadata difference record to the second attribute value, such as updating it to 0 or false. At the same time, the peer metadata change notification time of the second metadata difference record is updated to the generation time of the metadata change notification.

[0110] In practical applications, the metadata change notification carries the interval identifier of the second target sub-interval corresponding to the changed data; similarly, the peer node determines the corresponding second target sub-interval among multiple second sub-intervals based on the interval identifier carried in the metadata change notification; then, it updates the peer metadata change attribute and peer metadata change notification time of the second metadata difference record corresponding to the second target sub-interval.

[0111] In this embodiment, when data changes, updating the first metadata difference record by the local node enables the local node to record data changes in real time. By sending a metadata change notification to the peer node, the peer node can modify the peer metadata change attribute and peer metadata change notification time of the second metadata difference record. This allows the peer node to quickly determine whether data changes have occurred on the local node based on the peer metadata change attribute of the second metadata difference record when it needs to detect data changes between nodes. Furthermore, based on the peer metadata change notification time of the second metadata difference record, the peer node can quickly retrieve the corresponding change log from the local node to update the second metadata difference record.

[0112] In one exemplary embodiment, such as Figure 4 As shown, step S304 above, which sends a metadata change notification to the peer node, specifically includes the following steps:

[0113] Step S402: If the local node detects that the peer node is offline during the process of sending the metadata change notification, it marks the metadata change notification.

[0114] Specifically, in complex distributed networks, it is normal for nodes to go offline occasionally. Therefore, if a local node detects that a peer node is offline while sending a metadata change notification to the peer node, the local node will suspend sending the metadata change notification, mark it, and then store it in the local node's first database.

[0115] Step S404: When the peer node comes back online, it sends a change notification request to the local node.

[0116] Specifically, when a peer node re-enters the network after going offline, it sends a change notification request to the local node.

[0117] Step S406: In response to the change notification retrieval request, the local node sends the marked metadata change notification to the peer node.

[0118] Specifically, in response to a change notification retrieval request, the local node retrieves the tagged metadata change notification from its first database and sends it to the peer node.

[0119] In step S408, the peer node, in response to the marked metadata change notification, updates the peer metadata change attribute of the second metadata difference record to the second attribute value, updates the peer metadata change notification time of the second metadata difference record, and returns a first update response message to the local node.

[0120] Specifically, in response to the marked metadata change notification, the peer node updates the peer metadata change attribute and peer metadata change notification time of the second metadata difference record, and returns a first update response message to the local node after the update is completed, so as to inform the local node that the corresponding update has been completed.

[0121] In step S410, the local node, in response to the first update response message, cancels the marking of the marked metadata change notification.

[0122] Specifically, in response to the first update response message, the local node cancels the marking of the marked metadata change notification.

[0123] In practical applications, if a peer node initiates other business requests to this node before obtaining the marked metadata change notification, this node will reject the business requests and proceed directly to the peer node to obtain the marked metadata change notification.

[0124] In this embodiment, by marking the change notification retrieval request, it is possible to identify the change notification retrieval request that was not successfully sent, thereby ensuring that even if the peer node is offline, the change notification retrieval request that was not successfully sent can be resent to the peer node after the peer node comes back online.

[0125] In an exemplary embodiment, after determining the data that has undergone data change as changed data in step S302 above, the method further includes the following steps: generating and storing a change log of the changed data.

[0126] Specifically, after determining the changed data, the local node also needs to generate and store the change log for that data.

[0127] In practical applications, since nodes manage each change log according to each second sub-interval, after the local node generates a change log for the changed data, it needs to determine the target change log set corresponding to the change log from multiple change log sets based on the target feature value of the metadata of the changed data, and then add the change log to the target change log set.

[0128] like Figure 5 As shown, the inter-node data change detection method provided in this application further includes the following steps, used to periodically send change logs to the peer node:

[0129] Step S502: If the local metadata change attribute of the first metadata difference record is the second attribute value, the local node retrieves the corresponding second target change log from each change log stored in the local node. The log generation time of the second target change log is later than or equal to the local metadata change notification time of the first metadata difference record. The second target change log is then sent to the peer node.

[0130] Specifically, the local node periodically checks the local metadata change attribute of the first metadata difference record. If the local metadata change attribute of the first metadata difference record is the second attribute value, such as 0 or false, then it retrieves the stored change logs from the local node's first database, and retrieves the corresponding change log whose log generation time is later than or equal to the local metadata change notification time of the first metadata difference record from the stored change logs, as the second target change log, and then sends the second target change log to the peer node.

[0131] In practical applications, since the nodes manage the metadata information and change logs of the difference metadata in groups according to each second sub-interval, the local node periodically traverses each first metadata difference information record. If the local metadata change attribute of the traversed first metadata difference information record is the second attribute value, the local node determines the second sub-interval corresponding to the traversed first metadata difference information record as the third target sub-interval, and obtains the change log set corresponding to the third target sub-interval from the local node's first database, and then obtains the second target change log from it and sends it to the peer node.

[0132] Step S504: The peer node updates the second metadata difference record according to the second target change log, updates the peer metadata change attribute of the second metadata difference record to the first attribute value, and returns a second update response message to the local node.

[0133] Specifically, the peer node updates the second metadata difference record based on the metadata information of the corresponding changed data in the second target change log. At the same time, it updates the peer metadata change attribute of the second metadata difference record to the first attribute value, such as 1 or true. After the update is completed, it returns a second update response message to the local node to inform the local node that the corresponding update has been completed.

[0134] In practical applications, when the peer node receives the second target change log, it first determines the corresponding third target sub-interval among multiple second sub-intervals based on the metadata information of the metadata of the corresponding change data in the second target change log. Then, the peer node updates the second metadata difference record corresponding to the third target sub-interval and the peer metadata change attribute of the second metadata difference record.

[0135] In step S504, the local node, in response to the second update response message, updates the local metadata change attribute of the first metadata difference record to the first attribute value.

[0136] Specifically, in response to the second update response message, the local node updates the local metadata change attribute of the first metadata difference record to the first attribute value, such as 1 or true.

[0137] In practical applications, if the local node receives a second update response message after sending the second target change log for the first metadata difference record it has traversed, then the local metadata change attribute of the first metadata difference record it has traversed will be updated to the first attribute value.

[0138] In this embodiment, on the one hand, the local node can ensure that the peer node can promptly perceive the data changes of the local node by periodically pushing update logs; on the other hand, by having the peer node update the peer metadata change attribute to the first attribute, and by having the local node update the local metadata change attribute to the first attribute, the accuracy of subsequent data change detection between nodes can be ensured.

[0139] To more clearly illustrate the method for detecting data changes between nodes provided in the embodiments of this application, a specific embodiment is given below for detailed description. However, it should be understood that the embodiments of this application are not limited thereto. In one exemplary embodiment, this application also provides a data difference detection and synchronization method, specifically including the following steps:

[0140] Step 1: The local node and the peer node negotiate grouping. For example... Figure 6 The diagram shown is a flowchart of step one.

[0141] Step S602: The local node sends a request to the peer node to obtain the feature value range of the target metadata feature.

[0142] In step S604, the peer node, in response to the feature value range acquisition request, returns the second feature value range to the local node.

[0143] Step S606: The local node obtains the global feature value range of the target metadata feature based on the union of the first feature value range and the second feature value range. The global feature value range is divided into multiple first sub-ranges. Based on the multiple first sub-ranges and the target feature values ​​of each first metadata under the target metadata feature, each first metadata is divided into multiple first metadata information sets.

[0144] Step S608: The local node generates multiple metadata comparison requests based on multiple first sub-intervals and multiple first metadata information sets, and sends each metadata comparison request to the peer node.

[0145] Step S610: In response to each metadata comparison request, the peer node compares the metadata identifier of the second metadata that falls into the first sub-interval carried in the metadata comparison request with the metadata identifiers carried in the metadata comparison request to obtain the difference metadata between the local node and the peer node under the metadata comparison request, stores the metadata information of the difference metadata under each metadata comparison request, obtains the second metadata difference record, and returns the metadata information of the difference metadata under each metadata comparison request to the local node.

[0146] Step S612: The local node divides the global feature value interval into multiple second sub-intervals. Based on the target feature values ​​in the metadata information of each difference metadata and the multiple second sub-intervals, the metadata information of each difference metadata is divided into multiple second metadata information sets, and the first metadata difference record corresponding to each second metadata information set is determined.

[0147] Step Two: When the local node needs to use data or metadata stored locally, it determines the data change detection result of the peer node and synchronizes the metadata difference records based on the data change detection result. For example... Figure 7 The diagram shown is a flowchart of step two.

[0148] Step S702: The local node obtains each first metadata difference record stored in the local node and traverses each first metadata difference record.

[0149] Step S704: For each first metadata difference record, the local node determines the data change detection result of the peer node under the first metadata difference record based on the record attribute information of the first metadata difference record.

[0150] Step S706: For the local node, if the peer metadata change attribute of the first metadata difference record is the first attribute value, then the data change detection result of the peer node is determined to be that the data has not changed; if the peer metadata change attribute of the first metadata difference record is the second attribute value, then the data change detection result of the peer node is determined to be that the data has changed.

[0151] Step S708: When the data change detection result of the peer node is a data change, the local node obtains the first target change log from each change log corresponding to the second metadata difference record corresponding to the first metadata difference record, where the log generation time is later than or equal to the peer metadata change notification time of the first metadata difference record. Based on the first target change log, the first metadata difference record is updated.

[0152] Step 3: When the local node sends data changes, it updates the first metadata difference record and the corresponding record attribute information. For example... Figure 8 The diagram shown is a flowchart of step three.

[0153] Step S802: If the local node detects a data change, it identifies the changed data as the changed data and generates a corresponding change log. Based on the metadata information of the changed data, it identifies and updates the first metadata difference record corresponding to the changed data, updates the local metadata change attribute of the first metadata difference record to the second attribute value, sends a metadata change notification to the peer node, and updates the local metadata change notification time of the first metadata difference record.

[0154] Step S804: In response to the metadata change notification, the peer node updates the peer metadata change attribute of the second metadata difference record corresponding to the first metadata difference record stored by the peer node to the second attribute value, and updates the peer metadata change notification time of the second metadata difference record.

[0155] Step S806: If the local node detects that the peer node is offline during the process of sending the metadata change notification, it marks the metadata change notification.

[0156] In step S808, the peer node sends a change notification retrieval request to the local node when it comes back online.

[0157] In step S810, the local node, in response to the change notification retrieval request, sends the marked metadata change notification to the peer node.

[0158] In step S812, the peer node, in response to the marked metadata change notification, updates the peer metadata change attribute of the second metadata difference record to the second attribute value, updates the peer metadata change notification time of the second metadata difference record, and returns a first update response message to the local node.

[0159] In step S814, the local node, in response to the first update response message, cancels the marking of the marked metadata change notification.

[0160] Step 4: The local node periodically pushes change logs to the peer node. For example... Figure 9 The diagram shown is a flowchart of step four.

[0161] Step S902: For each first metadata difference record, if the local metadata change attribute of the first metadata difference record is the second attribute value, then the local node retrieves the corresponding second target change log from each change log stored by the local node that corresponds to the first metadata difference record. The log generation time of the second target change log is later than or equal to the local metadata change notification time of the first metadata difference record. The second target change log is then sent to the peer node.

[0162] Step S904: The peer node updates the second metadata difference record corresponding to the first metadata difference record according to the second target change log, updates the peer metadata change attribute of the second metadata difference record to the first attribute value, and returns a second update response message to the local node.

[0163] In step S906, the local node, in response to the second update response message, updates the local metadata change attribute of the first metadata difference record to the first attribute value.

[0164] In this embodiment, by grouping and managing the metadata information of the difference metadata, the impact of some data changes on the global difference information is reduced, making it easier to manage PB-level data differences; dividing according to the continuous characteristics of the metadata can quickly locate the group corresponding to the metadata with a time complexity of O(1); adopting the method of full comparison first and incremental change update later greatly reduces the amount of comparison data and improves the efficiency of dual-end difference maintenance; by synchronizing the dirty status of the dual-end node difference backup set group, the consistency of the dual-end status is ensured, solving the real-time problem; at the same time, batch timed synchronization of difference data does not require fingerprint calculation, reducing the computational cost under frequent data changes, improving data transmission efficiency, and reducing network transmission costs; allowing the data user to actively pull the dirty group block information to meet the real-time requirements and ensure that the data used is in the latest state; data storage is decoupled from database type, does not depend on database logs, has low database migration cost, and users can flexibly choose a database suitable for their business characteristics.

[0165] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0166] Based on the same inventive concept, this application also provides an inter-node data change detection device for implementing the above-described inter-node data change detection method. The solution provided by this device is similar to the implementation described in the above-described method. Therefore, the specific limitations in one or more embodiments of the inter-node data change detection device provided below can be found in the limitations of the inter-node data change detection method described above, and will not be repeated here.

[0167] In one exemplary embodiment, such as Figure 10 As shown, a device for detecting data changes between nodes is provided, including: a difference record acquisition module 1002 and a data change detection module 1004, wherein:

[0168] The difference record acquisition module 1002 is used to acquire the first metadata difference record stored by the local node; the local node is any one of a plurality of nodes; the first metadata difference record is used to record the metadata information of the difference metadata between the local node and the peer node; the peer node is any one of the plurality of nodes other than the local node; the metadata information includes at least the metadata identifier of the corresponding metadata.

[0169] The data change detection module 1004 is used to determine the data change detection result of the peer node based on the record attribute information of the first metadata difference record. The record attribute information of each metadata difference record includes at least the peer metadata change attribute. The peer metadata change attribute is used to characterize whether the corresponding peer node has different metadata information that has not been synchronized to the corresponding local node. The peer node is used to notify the local node to update the peer metadata change attribute of the first metadata difference record when the peer node undergoes data change.

[0170] In an exemplary embodiment, the recorded attribute information also includes the peer metadata change notification time; the peer metadata change notification time is the time when the corresponding peer node last sent a metadata change notification to the corresponding local node.

[0171] The data change detection module 1004 is further configured to determine that the data change detection result of the peer node is no data change if the peer metadata change attribute of the first metadata difference record is a first attribute value; and to determine that the data change detection result of the peer node is data change if the peer metadata change attribute of the first metadata difference record is a second attribute value.

[0172] The inter-node data change detection device also includes a difference record synchronization module, which is used to obtain the first target change log from each change log stored by the peer node when the data change detection result of the peer node is a data change, and the corresponding log generation time is later than or equal to the peer metadata change notification time of the first metadata difference record; and update the first metadata difference record based on the first target change log.

[0173] In an exemplary embodiment, the recorded attribute information further includes a local metadata change attribute, a peer metadata change notification time, and a local metadata change notification time. The local metadata change attribute is used to characterize whether the corresponding local node has metadata differences that have not been synchronized to the corresponding peer node. The peer metadata change notification time is the time when the corresponding peer node last sent a metadata change notification to the corresponding local node. The local metadata change notification time is the time when the corresponding local node last sent a metadata change notification to the corresponding peer node. Both the local metadata change attribute and the peer metadata change attribute are initialized to a first attribute value.

[0174] The inter-node data change detection device further includes a data change notification module, which, if a data change is detected at the local node, identifies the changed data as changed data, adds the metadata information of the changed data to a first metadata difference record, and updates the local metadata change attribute of the first metadata difference record to a second attribute value; sends a metadata change notification to the peer node, and updates the local metadata change notification time of the first metadata difference record; the peer node, in response to the metadata change notification, updates the peer metadata change attribute of the second metadata difference record stored by the peer node to the second attribute value, and updates the peer metadata change notification time of the second metadata difference record; the second metadata difference record is used to record the difference metadata between the local node and the peer node.

[0175] In an exemplary embodiment, the data change notification module is further configured to mark the metadata change notification if the peer node is detected to be offline during the sending of the metadata change notification; the peer node is further configured to send a change notification retrieval request to the local node when it comes back online; in response to the change notification retrieval request, the marked metadata change notification is sent to the peer node; the peer node is configured to update the peer metadata change attribute of the second metadata difference record to the second attribute value in response to the marked metadata change notification, and update the peer metadata change notification time of the second metadata difference record, and return a first update response message to the local node; in response to the first update response message, the marking of the marked metadata change notification is canceled.

[0176] In one exemplary embodiment, the data change notification module is also used to generate and store a change log of the changed data.

[0177] The difference record synchronization module is further configured to, if the local metadata change attribute of the first metadata difference record is the second attribute value, retrieve the corresponding second target change log from each change log stored on the local node whose log generation time is later than or equal to the local metadata change notification time of the first metadata difference record; send the second target change log to the peer node; the peer node is configured to update the second metadata difference record according to the second target change log, update the peer metadata change attribute of the second metadata difference record to the first attribute value, and return a second update response message to the local node; in response to the second update response message, update the local metadata change attribute of the first metadata difference record to the first attribute value.

[0178] In an exemplary embodiment, the inter-node data change detection device further includes a difference information determination module, configured to send a feature value interval acquisition request to the peer node; the feature value interval acquisition request carries a target metadata feature, which is any one of multiple metadata features; each first metadata stored by the local node has a corresponding first feature value interval under the target metadata feature, and each second metadata stored by the peer node has a corresponding second feature value interval under the target metadata feature; the peer node is configured to return a second feature value interval to the local node in response to the feature value interval acquisition request; a global feature value interval of the target metadata feature is obtained based on the union of the first feature value interval and the second feature value interval; the global feature value interval is divided into multiple first sub-intervals, and based on the multiple first sub-intervals and the target feature values ​​of each first metadata under the target metadata feature, the metadata information of each first metadata is divided into multiple first metadata information sets; each first metadata information set... Each set corresponds to a first sub-interval; based on multiple first sub-intervals and multiple sets of first metadata information, multiple metadata comparison requests are generated; each metadata comparison request carries a first sub-interval and metadata information of each first metadata in the first metadata information set corresponding to the first sub-interval; each metadata comparison request is sent to the peer node; the peer node, in response to each metadata comparison request, compares the metadata identifier of the second metadata that the corresponding target feature value falls into the first sub-interval carried by the metadata comparison request with the metadata identifiers carried by the metadata comparison request, obtains the difference metadata between the local node and the peer node under the metadata comparison request, stores the metadata information of the difference metadata under each metadata comparison request, obtains the second metadata difference record, and returns the metadata information of the difference metadata under each metadata comparison request to the local node; receives and stores the metadata information of the difference metadata under each metadata comparison request, obtains the first metadata difference record.

[0179] In an exemplary embodiment, the difference information determination module is further configured to divide the global feature value interval into multiple second sub-intervals; based on the target feature values ​​in the metadata information of each difference metadata and the multiple second sub-intervals, divide the metadata information of each difference metadata into multiple second metadata information sets; each second metadata information set corresponds to a second sub-interval; and determine the first metadata difference record corresponding to each second metadata information set.

[0180] Each module in the aforementioned inter-node data change detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0181] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores node data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a method for detecting data changes between nodes.

[0182] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0183] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0184] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0185] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0186] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0187] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0188] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting data changes between nodes, characterized in that, The method is applied to a local node among multiple nodes, where the local node is any one of the multiple nodes; the method includes: The system retrieves a first metadata difference record stored on the local node. This first metadata difference record records metadata information about the differences between the local node and the peer node. The peer node is any node among the plurality of nodes other than the local node. The metadata information includes at least a metadata identifier for the corresponding metadata. The peer node sends a metadata change notification to the local node when data changes occur on the peer node. In response to the metadata change notification, the local node updates the peer metadata change attribute and the peer metadata change notification time of the first metadata difference record. The peer metadata change attribute indicates whether the corresponding peer node has metadata information that has not been synchronized to the corresponding local node's metadata difference record. The peer metadata change notification time is the time when the corresponding peer node last sent a metadata change notification to the corresponding local node. Based on the peer metadata change attributes of the first metadata difference record, determine the data change detection result of the peer node; If the data change detection result of the peer node is a data change, a first target change log is obtained from each change log stored by the peer node; the first target change log is a change log whose corresponding log generation time is later than or equal to the peer metadata change notification time of the first metadata difference record; Based on the first target change log, update the first metadata difference record.

2. The method according to claim 1, characterized in that, The step of determining the data change detection result of the peer node based on the peer metadata change attributes recorded in the first metadata difference record includes: If the peer metadata change attribute of the first metadata difference record is the first attribute value, then the data change detection result of the peer node is determined to be that the data has not changed. If the peer metadata change attribute of the first metadata difference record is the second attribute value, then the data change detection result of the peer node is determined to be a data change.

3. The method according to claim 1, characterized in that, The method further includes: If a data change is detected at the local node, the data with the changed data is identified as changed data. The first metadata difference record is updated based on the metadata information of the changed data, and the local metadata change attribute of the first metadata difference record is updated to the second attribute value. The local metadata change attribute is used to characterize whether the corresponding local node has different metadata information that has not been synchronized to the corresponding peer node's metadata difference record. The local metadata change attribute is initialized to the first attribute value. The local node sends a metadata change notification to the peer node and updates the local metadata change notification time of the first metadata difference record. In response to the metadata change notification, the peer node updates the peer metadata change attribute of the second metadata difference record stored by the peer node to the second attribute value and updates the peer metadata change notification time of the second metadata difference record. The second metadata difference record records the difference metadata between the local node and the peer node. The peer metadata change attribute is initialized to the first attribute value. The local metadata change notification time represents the time when the corresponding peer node last sent a metadata change notification to the corresponding local node.

4. The method according to claim 3, characterized in that, Sending the metadata change notification to the peer node includes: If the peer node is detected to be offline during the sending of the metadata change notification, the metadata change notification is marked; the peer node is also used to send a change notification retrieval request to the local node when it comes back online; In response to the change notification retrieval request, the marked metadata change notification is sent to the peer node; the peer node is configured to, in response to the marked metadata change notification, update the peer metadata change attribute of the second metadata difference record to the second attribute value, update the peer metadata change notification time of the second metadata difference record, and return a first update response message to the local node; In response to the first update response message, the tagging of the tagged metadata change notification is removed.

5. The method according to claim 3, characterized in that, After identifying the data that has undergone changes as changed data, the following is also included: Generate and store a change log for the changed data; The method further includes: If the local metadata change attribute of the first metadata difference record is the second attribute value, then obtain the second target change log from each change log stored in the local node whose log generation time is later than or equal to the local metadata change notification time of the first metadata difference record. The second target change log is sent to the peer node; the peer node is used to update the second metadata difference record according to the second target change log, update the peer metadata change attribute of the second metadata difference record to the first attribute value, and return a second update response message to the local node; In response to the second update response message, the local metadata change attribute of the first metadata difference record is updated to the first attribute value.

6. The method according to any one of claims 1 to 5, characterized in that, The first metadata difference record is obtained in the following way: A feature value range acquisition request is sent to the peer node; the feature value range acquisition request carries a target metadata feature, which is any one of a plurality of metadata features; each first metadata stored by the local node has a corresponding first feature value range under the target metadata feature, and each second metadata stored by the peer node has a corresponding second feature value range under the target metadata feature; The peer node is configured to return the second feature value range to the local node in response to the feature value range acquisition request; The global feature value range of the target metadata feature is obtained by the union of the first feature value range and the second feature value range. The global feature value interval is divided into multiple first sub-intervals. Based on the multiple first sub-intervals and the target feature values ​​of each first metadata under the target metadata feature, the metadata information of each first metadata is divided into multiple first metadata information sets. Each set of first metadata information corresponds to a first sub-interval; Based on the plurality of first sub-intervals and the plurality of first metadata information sets, generate a plurality of metadata comparison requests; Each metadata comparison request carries a first sub-interval and the metadata information in the first metadata information set corresponding to the first sub-interval; Send each metadata comparison request to the peer node; the peer node is configured to respond to each metadata comparison request by comparing the metadata identifier of the second metadata whose corresponding target feature value falls into the first sub-interval carried by the metadata comparison request with each metadata identifier carried by the metadata comparison request, to obtain the difference metadata between the local node and the peer node under the metadata comparison request, to store the metadata information of the difference metadata under each metadata comparison request, to obtain the second metadata difference record, and to return the metadata information of the difference metadata under each metadata comparison request to the local node; Receive and store the metadata information of the difference metadata under each metadata comparison request to obtain the first metadata difference record.

7. The method according to claim 6, characterized in that, The metadata information also includes the target feature values ​​of the corresponding metadata; The step of receiving and storing the metadata information of the difference metadata under each metadata comparison request to obtain the first metadata difference record includes: The global feature value interval is divided into multiple second sub-intervals; Based on the target feature values ​​in the metadata information of the multiple second sub-intervals and each differential metadata, the metadata information of each differential metadata is divided into multiple second metadata information sets; each second metadata information set corresponds to a second sub-interval. Identify the first metadata difference record corresponding to each second metadata information set.

8. A device for detecting data changes between nodes, characterized in that, The device includes: A difference record acquisition module is used to acquire a first metadata difference record stored on the local node; the local node is any one of multiple nodes; the first metadata difference record is used to record metadata information of the difference metadata between the local node and the peer node; the peer node is any one of the multiple nodes other than the local node; the metadata information includes at least the metadata identifier of the corresponding metadata; the peer node is used to send a metadata change notification to the local node when the peer node undergoes data changes; the local node is used to update the peer metadata change attribute and peer metadata change notification time of the first metadata difference record in response to the metadata change notification; the peer metadata change attribute is used to characterize whether the corresponding peer node has difference metadata whose corresponding metadata information has not been synchronized to the corresponding local node's metadata difference record; the peer metadata change notification time is the time when the corresponding peer node last sent a metadata change notification to the corresponding local node. The data change detection module is used to determine the data change detection result of the peer node based on the record attribute information of the first metadata difference record; The difference record synchronization module is used to obtain a first target change log from each change log stored by the peer node when the data change detection result of the peer node is a data change; the first target change log is a change log whose corresponding log generation time is later than or equal to the peer metadata change notification time of the first metadata difference record; and to update the first metadata difference record based on the first target change log.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images