A method and device for automatically making up for lost data

Abstract

The present application relates to a method and device for automatically making up for lost data, the method comprising the steps of: obtaining the current CRC value, incremental data and changed CRC value of the managed device, the changed CRC value being the full data CRC value after the service node data of the managed device is changed; the network management local CRC value being the full data CRC value reported by the managed device when it was successfully synchronized last time; comparing the current CRC value with the network management local CRC value, if they are inconsistent, replying the network management local CRC value to the managed device as the expected CRC value, the expected CRC value being used to enable the managed device to search for lost data according to the expected CRC value and realize incremental synchronization between the network management and the device according to the search result. The method of the present invention can solve the problems in the related art and improve the synchronization efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of communication system data synchronization, and in particular to a method and device for automatically making up for lost data. Background Art

[0002] At present, in the communication system network, when the network management and equipment data are synchronized, the traditional full data synchronization method is reliable but has a large resource load. The existing incremental data method has high synchronization efficiency but is prone to network failures or application vulnerabilities, resulting in loss of data to be synchronized and inconsistencies.

[0003] In the related technology, when the current network management finds inconsistent data, it uses polling to check and synchronize the whole data, which is inefficient; the network management needs to accurately retrieve the incremental lost data from the device, or the device actively reports the predictable lost data during the disconnection period. In addition, for large-scale discrete and variable status data, the calculation of CRC in the related technology is inefficient. Summary of the invention

[0004] The embodiment of the present invention provides a method and device for automatically making up for lost data, so as to solve the problems in the related art and improve the synchronization efficiency.

[0005] The embodiment of the present invention provides a method for automatically making up for lost data, characterized in that it comprises the steps of:

[0006] Obtain the current CRC value, incremental data, and changed CRC value of the managed device, where the changed CRC value is the full data CRC value after the service node data of the managed device is changed;

[0007] Compare the current CRC value with the local CRC value of the network management, and if they are inconsistent, reply the local CRC value of the network management as the expected CRC value to the managed device, and the expected CRC value is used to enable the managed device to search for lost data according to the expected CRC value and realize incremental synchronization between the network management and the device according to the search result;

[0008] The local CRC value of the network management is the CRC value of the full data successfully reported by the managed device during the last synchronization.

[0009] In some embodiments, the step is also included: comparing the current CRC value with the network manager local CRC value, if they are consistent, updating the network manager local CRC value according to the changed CRC value and replying the updated network manager local CRC value as the expected CRC value to the managed device.

[0010] In some embodiments, the current CRC value, incremental data, and changed CRC value of the managed device are recorded and stored in a historical change data warehouse according to the time when the node data change occurred;

[0011] The managed device searches for lost data according to the expected CRC value and realizes incremental synchronization between the network management and the device according to the search result, including the steps of:

[0012] If the expected CRC value is found in the historical change data warehouse, all change data from the time point corresponding to the expected CRC value to the latest recorded time point are merged and reported to the network management;

[0013] If the expected CRC value is not found in the historical change data warehouse, the network manager is notified of the failure of the make-up call so that the network manager triggers full data synchronization.

[0014] In some embodiments, before searching for missing data according to the expected CRC value, the managed device also compares the expected CRC value with the latest changed CRC value. If they are inconsistent, it searches for missing data. If they are consistent, it notifies the network manager to end synchronization confirmation.

[0015] In some embodiments, the CRC value of the full data of the service node data of the managed device after the data is changed is calculated based on the CRC value of the node after the data change, the CRC value of all prefix node sets of the node, and the CRC value of all suffix node sets of the node.

[0016] The embodiment of the present invention further provides a device for automatically making up for lost data, characterized in that it comprises:

[0017] The make-up call information generation module is used to:

[0018] Obtain the current CRC value, incremental data, and changed CRC value of the managed device, where the changed CRC value is the full data CRC value after the service node data of the managed device is changed;

[0019] Compare the current CRC value with the local CRC value of the network management, and if they are inconsistent, reply the local CRC value of the network management as the expected CRC value to the managed device;

[0020] Missing data search module, which is used to:

[0021] Searching for lost data according to the expected CRC value and implementing incremental synchronization between the network management and the device according to the search result;

[0022] The local CRC value of the network management is the CRC value of the full data successfully reported by the managed device during the last synchronization.

[0023] In some embodiments, the supplementary call information generation module is further used to:

[0024] Compare the current CRC value with the network management local CRC value. If they are consistent, update the network management local CRC value according to the changed CRC value and reply the updated network management local CRC value as the expected CRC value to the managed device.

[0025] In some embodiments, the current CRC value, incremental data, and changed CRC value of the managed device are recorded and stored in a historical change data warehouse according to the time when the node data change occurred;

[0026] The lost data search module is also used for:

[0027] If the expected CRC value is found in the historical change data warehouse, all change data from the time point corresponding to the expected CRC value to the latest recorded time point are merged and reported to the network management;

[0028] If the expected CRC value is not found in the historical change data warehouse, the network manager is notified of the failure of the make-up call so that the network manager triggers full data synchronization.

[0029] In some embodiments, the lost data search module is further used to:

[0030] Before searching for lost data according to the expected CRC value, the expected CRC value is also compared with the latest changed CRC value. If they are inconsistent, the lost data is searched. If they are consistent, the network manager is notified to end the synchronization confirmation.

[0031] In some embodiments, the CRC value of the full data after the service node data of the managed device is changed is calculated based on the CRC value of the node after the data is changed, the CRC value of all prefix node sets of the node, and the CRC value of all suffix node sets of the node. The technical solution provided by the present invention has the following beneficial effects:

[0032] The embodiment of the present invention provides a method and device for automatically making up for lost data, so that when the network management and the device perform incremental synchronization of data, the network management actively discovers CRC inconsistency and provides automatic making up for the lost data in response to the situation, thereby solving the problem of the device actively reporting predictable lost data when the synchronization channel is disconnected, and effectively improving real-time performance and update efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1A schematic diagram of a flow chart of a method for automatically making up for lost data provided by an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of parallel calculation of CRC values ​​provided by an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of a device for automatically making up for lost data provided by an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of a channel reconstruction self-healing system provided by an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of the structure of a device for automatically making up for lost data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] like Figure 1 As shown, the embodiment of the present invention provides a method for automatically making up for lost data to solve the problems in the related art and improve synchronization efficiency. The steps include:

[0041] S100: Obtain the current CRC value, incremental data, and changed CRC value of the managed device;

[0042] S200: Compare the current CRC value with the local CRC value of the network manager. If they are inconsistent, reply the local CRC value of the network manager as the expected CRC value to the managed device. The expected CRC value is used to enable the managed device to search for lost data according to the expected CRC value and achieve incremental synchronization between the network manager and the device based on the search result.

[0043] It should be noted that the full data CRC value of a node refers to the cyclic redundancy check (CRC) value of the node. The changed CRC value is the full data CRC value of the service node data of the managed device after the change, and the network management local CRC value is the full data CRC value reported by the device when it was successfully synchronized last time.

[0044] It is understandable that when the node data of a service node on a device changes, the device will report the current CRC value of the node with the changed data (the CRC value before the data change stored in the device), the incremental data, and the changed CRC value (the CRC value calculated and updated after the data change) to the network manager.

[0045] In view of the problem that the current network management uses polling checks and full synchronization when discovering data inconsistencies, which leads to low efficiency, this embodiment proposes a method in which the network management actively reports the make-up call information so that the device can accurately retrieve the incremental lost data from the device as needed, solves the problem that the device actively reports the predictable lost data when the synchronization channel is broken, and effectively improves the real-time performance and update efficiency.

[0046] In some embodiments, in S200, the current CRC value is compared with the network management local CRC value. If they are consistent, the network management local CRC value is updated according to the changed CRC value and the updated network management local CRC value is replied to the managed device as the expected CRC value.

[0047] It is understandable that after the network manager receives the change data packet sent by the device (including the current CRC value, incremental data, and the changed CRC value), it can be processed in two situations. The first is to compare the current CRC value with the local CRC value of the network manager. If it is inconsistent, the change data packet is discarded and the expected CRC value is given (the expected CRC value at this time is the local CRC value of the network manager). The second is to compare the current CRC value with the local CRC value of the network manager. If it is consistent, the change data packet is processed, the local CRC value of the network manager is updated, and the device is replied with the updated CRC value. The network manager will reply with the expected CRC value for all the change data packets sent by the device, which can be understood as the expected next CRC request packet (when the network manager processes the incremental data of the device, the device may have new changes).

[0048] In some embodiments, in step S200, the managed device searches for lost data according to the expected CRC value and implements incremental synchronization between the network management and the device according to the search result, including the steps of:

[0049] S210: If the expected CRC value is found in the historical change data warehouse, all change data from the time point corresponding to the expected CRC value to the latest recorded time point are merged and reported to the network management;

[0050] S211: If the expected CRC value is not found in the historical change data warehouse, the network manager is notified of the failure of the make-up call so that the network manager triggers full data synchronization.

[0051] It should be noted that the current CRC value, incremental data, and changed CRC value of the managed device are recorded and stored in the historical change data warehouse according to the time when the node data change occurred. It can be understood that after each data change, the device will store the changed data blocks (including the incremental data of each change of the corresponding data change node, the current CRC value, and the changed CRC value) in order according to the timestamp of the change, forming a historical warehouse composed of changed data blocks (i.e., historical change data warehouse).

[0052] In this embodiment, the device finds the timestamp corresponding to the expected CRC value, that is, the timestamp of the starting point of the lost data. All the change data from this timestamp to the timestamp of the latest change data are merged and packaged and reported to the network manager, so that the network manager and the device can perform incremental synchronization updates. If the device does not find the timestamp corresponding to the expected CRC value, and determines that there is lost data but it cannot be traced back, it will notify the network manager that the call for compensation has failed, and the network manager will initiate full data synchronization of the device. Full data synchronization means that the network manager retrieves all current data from the device and obtains the CRC value after the device changes, directly overwriting the network manager's data and local CRC value. The device will then clean up the historical change data warehouse.

[0053] In some embodiments, before the managed device searches for missing data according to the expected CRC value, it also compares the expected CRC value with the latest changed CRC value. If they are inconsistent, it searches for missing data. If they are consistent, it notifies the network manager to end synchronization confirmation.

[0054] It is understandable that the device compares the expected CRC value replied by the network manager with the CRC value of the device after the current change. If they are consistent, it is determined that there is no missing data, and the network manager is notified to end the confirmation. If they are inconsistent, it is determined that there is missing data, and after determining that there is missing data, step S210 of searching for missing data is performed.

[0055] Preferably, the CRC value of the full data of the service node data of the managed device after the data is changed is calculated based on the CRC value of the node with the data change, the CRC value of all prefix node sets of the node, and the CRC value of all suffix node sets of the node.

[0056] It can be understood that when the device data changes, the digital label CRC of the full data is calculated. The device abstracts each service data into a node (Node) to assemble the data. Each service node maintains three CRC values, including the CRC value of the node, the CRC value of all prefix nodes in sequence (that is, the CRC value of all prefix node sets of the node, which can be recorded as CRCL) and the CRC value of all suffix nodes in sequence (that is, the CRC value of all suffix node sets of the node, which can be recorded as CRCR). When the Node data changes, the CRC value of the node after the change is first calculated according to the standard CRC32 algorithm (recorded as CRC'), and the CRC value of the full data is calculated according to the combination of CRCL+CRC'+CRCR (recorded as CRC-SUM), and CRC-SUM is the CRC value after the change reported to the network management. Preferably, the calculation of CRCL, CRCR and the calculation of CRC-SUM can be processed in parallel, where CRC-SUM is required for reporting the changed data to the network management, and CRCL and CRCR are required for the background calculation of the next CRC-SUM calculation preparation.

[0057] This embodiment uses an iterative CRC calculation method to solve the problem of low efficiency in CRC value traversal calculation for large-scale discrete and changeable state data, which can effectively improve the calculation efficiency of the changed node.

[0058] In some embodiments, the device maintains three CRC values ​​for each node, including the CRC value of the node (denoted as CRC m , m is the node number), the CRC value of all prefix node sets of this node (recorded as CRCL1-(m-1)), the CRC value of all suffix node sets of this node (recorded as CRCR(m+1) -n, n is the total number of nodes); when the data of the mth node changes, the CRC that depends on its data will change, including the mth node CRC, the 1st to (m-1)th node CRCL, and the (m+1)th node to the nth node CRCR, a total of n CRC changes; since the CRCL and CRCR of the mth node itself do not change, the full data CRC-SUM can be quickly calculated for reporting to the network management; the CRCL or CRCR of other nodes can be calculated in parallel in the background.

[0059] like Figure 2 As shown in a specific embodiment, if the data of Node 3 is changed, the CRC3 value of the node after the data change is calculated first, and then the CRCL value is calculated according to the CRCL value. 1-2 +CRC3+CRCR 4-1000Combine and calculate CRC-SUM, and report the changed CRC value of the final node 3 to the network management with CRC-SUM. Among them, the first node has no CRCL value, and the tail node has no CRCR value, so the initial values ​​given according to the characteristics of the CRC32 algorithm are all FF. At the same time, because Node3 refreshes the CRC3 value of this node, it affects the CRCR values ​​of the prefix nodes Node1 and Node2, as well as the CRCL values ​​of the suffix nodes Node4 to Node1000. In parallel, the device traverses the Left node set forward to refresh the CRCR values ​​of all nodes containing CRC3, and traverses the Right node set backward to refresh the CRCL values ​​of all nodes containing CRC3.

[0060] like Figure 3 As shown, in a specific embodiment, the specific process of the method for automatically making up for lost data to achieve incremental synchronization is as follows:

[0061] When the device generates changed data, it reports the changed data block CRC1 (the current CRC value of this update) + increase A (this incremental data) + CRC2 (the CRC value after this change);

[0062] The network management compares the local CRC value (CRC1) with the current CRC value (CRC1) of the device and sends the expected CRC value (CRC2);

[0063] The device compares the changed CRC value (CRC2) with the expected CRC value (CRC2) sent by the network manager, determines that the data is consistent, and notifies the network manager to end the synchronization confirmation.

[0064] After that, the device generates a changed data block CRC2+addition B+CRC3, but does not report it to the network management, and the device generates a changed data block CRC3+deletion B+CRC2, and reports the wrong data to the network management;

[0065] Before the NMS confirms, the device generates the changed data block CRC2+modified A+CRC4 and the changed data block CRC4+added C+CRC5, but does not report to the NMS;

[0066] The network management compares the local CRC value (CRC2) with the current CRC value (CRC3) of the device and finds that they are inconsistent, so it sends the expected CRC value (CRC2);

[0067] The device reversely finds the first expected CRC2 and removes the redundant changes of adding B and deleting B. The device combines the changed data block CRC2+modified A+CRC4 and the changed data block CRC4+added C+CRC5 into a package and reports them.

[0068] The network manager updates the local CRC value with CRC5 and replies with the expected CRC value (CRC5). The device notifies the network manager to end the synchronization confirmation.

[0069] It should be noted that, in this embodiment, the sequence numbers 1 to 5 in CRC1 to CRC5 represent the order in which the changes occur.

[0070] In this embodiment, if the device finds multiple corresponding timestamps according to the expected CRC value, the timestamp corresponding to the first CRC value after the most recent successful synchronization is taken as the starting point of the lost data. Preferably, when there is redundant increment, the redundant changed data block is directly removed.

[0071] Furthermore, if Figure 4 As shown, the method of automatically replenishing lost data can be used to achieve channel reconstruction and self-healing. Specifically, the device continuously generates change data, and after the change of CRC2+adding B+CRC3, the synchronous channel is disconnected, and the data cannot be reported to the network management; the device merges and packages the change data from the channel disconnection to the recovery period, and reports CRC2+modifying A+adding C+CRC5 to the network management to achieve channel reconstruction; the network management confirms the data CRC5, and the channel reconstruction is completed to achieve self-healing.

[0072] like Figure 5 As shown, a device for automatically making up for lost data includes:

[0073] The make-up call information generation module is used to: obtain the current CRC value, incremental data and changed CRC value of the managed device; compare the current CRC value with the local CRC value of the network management, and if they are inconsistent, reply the local CRC value of the network management as the expected CRC value to the managed device;

[0074] A lost data search module, which is used to search for lost data according to the expected CRC value and realize incremental synchronization between the network management and the device according to the search result;

[0075] It should be noted that the changed CRC value is the CRC value of the full data after the service node data of the managed device is changed; the local CRC value of the network management is the CRC value of the full data reported by the managed device when it was successfully synchronized last time.

[0076] In some embodiments, the supplementary call information generation module is further used to:

[0077] Compare the current CRC value with the network management local CRC value. If they are consistent, update the network management local CRC value according to the changed CRC value and reply the updated network management local CRC value as the expected CRC value to the managed device.

[0078] In some embodiments, the current CRC value, incremental data, and changed CRC value of the managed device are recorded and stored in the historical change data warehouse according to the time when the node data change occurred;

[0079] The missing data lookup module is also used to:

[0080] If the expected CRC value is found in the historical change data warehouse, all change data from the time point corresponding to the expected CRC value to the latest recorded time point are merged and reported to the network management;

[0081] If the expected CRC value is not found in the historical change data warehouse, the network manager is notified of the failure of the make-up call so that the network manager triggers full data synchronization.

[0082] In some embodiments, the lost data search module is further used to:

[0083] Before searching for lost data according to the expected CRC value, the expected CRC value is also compared with the latest changed CRC value. If they are inconsistent, the lost data is searched. If they are consistent, the network manager is notified to end the synchronization confirmation.

[0084] In some embodiments, the CRC value of the full data after the service node data of the managed device is changed is calculated based on the CRC value of the node after the data change, the CRC value of all prefix node sets of the node, and the CRC value of all suffix node sets of the node.

[0085] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer-readable storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0086] The above are only specific implementations of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the embodiments of the present invention, and these modifications or replacements should be included in the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for automatically making up for lost data, characterized in that: It includes the steps of: Obtain the current CRC value, incremental data, and changed CRC value of the managed device, where the changed CRC value is the full data CRC value after the service node data of the managed device is changed; Compare the current CRC value with the local CRC value of the network management, and if they are inconsistent, reply the local CRC value of the network management as the expected CRC value to the managed device, and the expected CRC value is used to enable the managed device to search for lost data according to the expected CRC value and realize incremental synchronization between the network management and the device according to the search result; The local CRC value of the network management is the CRC value of the full data reported by the managed device during the last successful synchronization; Compare the current CRC value with the local CRC value of the network management. If they are consistent, update the local CRC value of the network management according to the changed CRC value and reply the updated local CRC value of the network management as the expected CRC value to the managed device; The current CRC value, incremental data and changed CRC value of the managed device are recorded and stored in the historical change data warehouse according to the time when the node data change occurs; The managed device searches for lost data according to the expected CRC value and realizes incremental synchronization between the network management and the device according to the search result, including the steps of: If the expected CRC value is found in the historical change data warehouse, all change data from the time point corresponding to the expected CRC value to the latest recorded time point are merged and reported to the network management; If the expected CRC value is not found in the historical change data warehouse, the network manager is notified of the failure of the make-up call so that the network manager triggers full data synchronization; The CRC value of the full data of the service node data of the managed device after the data is changed is calculated based on the CRC value of the node after the data change, the CRC value of all prefix node sets of the node, and the CRC value of all suffix node sets of the node.

2. A method for automatically making up for lost data as claimed in claim 1, characterized in that: Before searching for lost data according to the expected CRC value, the managed device also compares the expected CRC value with the latest changed CRC value. If they are inconsistent, it searches for lost data. If they are consistent, it notifies the network manager to end synchronization confirmation.

3. A device for automatically making up for lost data, characterized in that: It includes: The make-up call information generation module is used to: Obtain the current CRC value, incremental data, and changed CRC value of the managed device, where the changed CRC value is the full data CRC value after the service node data of the managed device is changed; Compare the current CRC value with the local CRC value of the network management, and if they are inconsistent, reply the local CRC value of the network management as the expected CRC value to the managed device; Missing data search module, which is used to: Searching for lost data according to the expected CRC value and implementing incremental synchronization between the network management and the device according to the search result; The local CRC value of the network management is the CRC value of the full data reported by the managed device during the last successful synchronization; The supplementary call information generation module is also used for: Compare the current CRC value with the local CRC value of the network management. If they are consistent, update the local CRC value of the network management according to the changed CRC value and reply the updated local CRC value of the network management as the expected CRC value to the managed device; The current CRC value, incremental data and changed CRC value of the managed device are recorded and stored in the historical change data warehouse according to the time when the node data change occurs; The lost data search module is also used for: If the expected CRC value is found in the historical change data warehouse, all change data from the time point corresponding to the expected CRC value to the latest recorded time point are merged and reported to the network management; If the expected CRC value is not found in the historical change data warehouse, the network manager is notified of the failure of the make-up call so that the network manager triggers full data synchronization; The CRC value of the full data of the service node data of the managed device after the data is changed is calculated based on the CRC value of the node after the data change, the CRC value of all prefix node sets of the node, and the CRC value of all suffix node sets of the node.

4. The device for automatically making up lost data according to claim 3, characterized in that: The lost data search module is also used for: Before searching for lost data according to the expected CRC value, the expected CRC value is also compared with the latest changed CRC value. If they are inconsistent, the lost data is searched. If they are consistent, the network manager is notified to end the synchronization confirmation.

Images