Data exchange system and method based on data compression identifier

Abstract

The present invention discloses a data exchange system and method based on a data compression identifier, which belongs to the field of data transmission technology. The technical problem to be solved is: how to effectively identify duplicate data, flexibly respond to data updates and easily realize data exchange during the data exchange process. It includes: a data exchange control module, which is used to filter out incremental data and updated local data as change data; a data fingerprint generation module, which is used to perform multi-layer hash calculations on the change data, extract feature vectors for the full data of each data structure, and use the feature vectors corresponding to the encrypted change data and the full data of various data structures as data fingerprints; a data exchange control module, which is used to encrypt and digitally sign the filtered data fingerprint set, and send the encrypted data fingerprint set to the target end; an incremental update module, which is used to decrypt and verify the encrypted data fingerprint set, update the local data of the target end based on the decryption result, and send a response to the source end.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a data exchange system and method based on a data compression identifier. Background Art

[0002] With the widespread adoption of big data applications, the demand for data exchange is growing. However, traditional data exchange methods often result in a large amount of redundant transmission, affecting efficiency. Although data deduplication and compression technologies already exist, these technologies have limitations, such as high complexity, high computational costs, and an inability to adapt to rapidly changing data environments.

[0003] How to effectively identify duplicate data, flexibly respond to data updates and easily implement data exchange during the data exchange process is a technical problem that needs to be solved. Summary of the Invention

[0004] The technical task of the present invention is to address the above shortcomings and provide a data exchange system and method based on data compression identifiers to solve the technical problems of how to effectively identify duplicate data, flexibly respond to data updates and easily realize data exchange during the data exchange process.

[0005] In a first aspect, the present invention provides a data exchange system based on a data compression identifier, which is used to implement data exchange between a source end and a target end, and includes a plug-in data fingerprint generation module, a data exchange control module, and an incremental update module;

[0006] The data exchange control module is used to analyze the data at the source end, filter out the incremental data and the updated local data, and use the incremental data and the updated local data as the changed data;

[0007] The data fingerprint generation module is used to support user configuration of multiple hash algorithms and multiple feature vector extraction algorithms, and is used to perform multi-layer hash calculations on the change data based on the hash algorithm to obtain encrypted change data. For the full data of various data structures, feature vectors are extracted from the full data of each data structure using a matching feature vector extraction algorithm, and the feature vectors corresponding to the encrypted change data and the full data of various data structures are used as data fingerprints to construct a data fingerprint set;

[0008] The data exchange control module is used to screen the data fingerprints in the data fingerprint set, remove duplicate data fingerprints through data screening, encrypt and digitally sign the screened data fingerprint set, and send the encrypted data fingerprint set to the target end;

[0009] For the encrypted data fingerprint set received by the target end, the incremental update module is used to decrypt and verify the encrypted data fingerprint set, obtain a decryption result including the changed data and the feature vectors corresponding to various data structures, and update the local data of the target end based on the decryption result. If the update is successful, a response of successful reception is sent to the source end through the data exchange control module. If the update fails, a response of failed reception is sent to the source end through the data exchange control module.

[0010] Preferably, the data exchange control module is configured to filter out incremental data and updated local data through Rsync or a similar differential algorithm.

[0011] Preferably, for the full amount of structured or semi-structured data, the data fingerprint generation module is used to identify the key fields or data patterns of the data as feature vectors; for unstructured data, the data fingerprint generation module is used to calculate the approximate features of the data as feature vectors through local sensitive hashing.

[0012] Preferably, for each full amount of data at the source end, the data fingerprint generation module is used to set a priority for each full amount of data;

[0013] The data exchange control module is configured with an adaptive transmission mode, which is used to detect the current network transmission status between the source end and the target end based on the adaptive transmission mode. If the current network transmission status is lower than the threshold, the data exchange control module is used to prioritize the transmission of the encrypted feature vector corresponding to the high-priority full data or the transmission of the encrypted change data, wherein the network transmission status includes the current network bandwidth and latency.

[0014] Preferably, the data exchange control module is configured to perform data screening on the data fingerprints in the data fingerprint set through a Bloom filter.

[0015] Preferably, the data exchange control module is configured with a log recording and auditing mechanism. Based on the log recording and auditing, the data exchange control module is used to screen the key data of the full data and changed data of the source end, use the screened key data as the data summary, and record the operations in the data fingerprint generation module, the data exchange control module and the incremental update module in the operation log to form data operation information, and construct the log recording information based on the data summary and the data operation information, wherein the data operation information includes a timestamp and an operation result, and the operation result includes two types: successful reception at the receiving end and failed reception at the receiving end.

[0016] Preferably, an ETL tool is configured in the data exchange control module, which is used to screen the data fingerprints in the data fingerprint set, encrypt the screened data fingerprint set through the TLS / SSL encryption method, send the encrypted data fingerprint set to the target end, and store the encrypted data fingerprint set.

[0017] Preferably, the data exchange system further includes a compatible interface, which is used to be compatible with components developed by a third party, including components for implementing data compression, components for implementing transmission optimization, and components for implementing security enhancement.

[0018] Preferably, the data exchange system further includes a dynamic scaling management module, which is used to support users in adjusting resource allocation of computing nodes where the source and target ends are located, including adding computing nodes, expanding the storage space of each computing node, and optimizing the allocation of network bandwidth between the source and target ends.

[0019] In a second aspect, the present invention provides a data exchange method based on a data compression identifier, for implementing data exchange between a source end and a target end through a data exchange system based on a data compression identifier as described in any one of the first aspects, comprising the following steps:

[0020] The data exchange control module analyzes the data on the source end, filters out the incremental data and updated local data, and uses the incremental data and updated local data as the changed data;

[0021] The user configures multiple hash algorithms and multiple feature vector extraction algorithms in the data fingerprint generation module. Based on the hash algorithm, multi-layer hash calculations are performed on the change data to obtain encrypted change data. For the full data of various data structures, feature vectors are extracted from the full data of each data structure using the matching feature vector extraction algorithm. The feature vectors corresponding to the encrypted change data and the full data of various data structures are used as data fingerprints to construct a data fingerprint set.

[0022] The data exchange control module screens the data fingerprints in the data fingerprint set, removes duplicate data fingerprints through data screening, encrypts and digitally signs the screened data fingerprint set, and sends the encrypted data fingerprint set to the target end;

[0023] For the encrypted data fingerprint set received by the target end, the encrypted data fingerprint set is decrypted and verified through the incremental update module to obtain the decryption result including the changed data and the corresponding feature vectors of various data structures, and the local data of the target end is updated based on the decryption result. If the update is successful, a response of successful reception is sent to the source end through the data exchange control module. If the update fails, a response of failed reception is sent to the source end through the data exchange control module.

[0024] The data exchange system and method based on data compression identifier of the present invention have the following advantages:

[0025] 1. Filter incremental data and updated local data as change data, extract feature vectors corresponding to the full data of various data structures, perform hash calculations on the change data and feature vectors as data fingerprints, and send the data fingerprints to the target end. Compared with traditional data transmission, this reduces redundant data transmission, speeds up data exchange, reduces network bandwidth pressure, and reduces the consumption of storage and computing resources.

[0026] 2. The method of filtering incremental data and updating local data and transmitting changed data ensures the real-time and accuracy of data;

[0027] 3. The system can be compatible with third-party developed components through compatible interfaces, which enhances the flexibility and applicability of the system;

[0028] 4. The system can automatically or manually adjust resource allocation through the dynamic scaling management module to ensure continuous high performance;

[0029] 5. Allows easy addition of user-configured hashing algorithms and feature vector extraction algorithms to adapt to changing needs and technological advancements. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] The present invention will be further described below with reference to the accompanying drawings.

[0032] Figure 1 This is a flowchart of a data exchange method based on a data compression identifier in Example 2. DETAILED DESCRIPTION

[0033] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments given are not intended to limit the present invention. Unless there is a conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0034] The embodiments of the present invention provide a data exchange system and method based on a data compression identifier, which are used to solve the technical problems of how to effectively identify duplicate data, flexibly respond to data updates, and easily implement data exchange during the data exchange process.

[0035] Example 1:

[0036] The present invention provides a data exchange system based on a data compression identifier, comprising a plug-in data fingerprint generation module, a data exchange control module and an incremental update module, for realizing data exchange between a source end and a target end.

[0037] The data exchange control module is used to analyze the data at the source end, filter out the incremental data and updated local data, and use the incremental data and updated local data as the changed data.

[0038] In this embodiment, the data exchange control module is used to filter out incremental data and updated local data through Rsync or a similar differential algorithm.

[0039] The data fingerprint generation module is used to support users to configure multiple hash algorithms and multiple feature vector extraction algorithms. It is used to perform multi-layer hash calculations on the change data based on the hash algorithm to obtain encrypted change data. For the full data of various data structures, the feature vector of each data structure is extracted through the matching feature vector extraction algorithm. The feature vectors corresponding to the encrypted change data and the full data of various data structures are used as data fingerprints to construct a data fingerprint set.

[0040] As a specific implementation of the data fingerprint generation module, this module is configured with three algorithms: SHA-256, CRC32, and LSH. In specific applications, a two-layer hash algorithm is used to encrypt changing data. The first round uses SHA-256 to ensure data integrity and security, while the second round uses the faster hash algorithm CRC32 to reduce the fingerprint code length, balancing security and efficiency. For full-scale structured and semi-structured data, key fields or data patterns are identified as feature vectors, and the fingerprint information is further refined through the combination of these features. For unstructured data, local sensitive hashing (LSH) is used to capture the approximate characteristics of the data as feature vectors.

[0041] Correspondingly, the data exchange control module is used to screen the data fingerprints in the data fingerprint set, remove duplicate data fingerprints through data screening, encrypt and digitally sign the screened data fingerprint set, and send the encrypted data fingerprint set to the target end.

[0042] As a specific implementation of the data exchange control module, the data exchange control module is configured with an ETL tool. The ETL tool is used to screen the data fingerprints in the data fingerprint set to reduce the actual transmission volume of subsequent data, and encrypt the screened data fingerprint set through the TLS / SSL encryption method to ensure the security of the data during transmission. The encrypted data fingerprint set is sent to the target end and stored to increase the risk defense against data leakage.

[0043] As a specific implementation of data screening, the data exchange control module is used to perform data screening on the data fingerprints in the data fingerprint set through a Bloom filter.

[0044] For the encrypted data fingerprint set received by the target end, the incremental update module is used to decrypt and verify the encrypted data fingerprint set, obtain a decryption result including the changed data and the feature vectors corresponding to various data structures, and update the local data of the target end based on the decryption result. If the update is successful, a response of successful reception is sent to the source end through the data exchange control module. If the update fails, a response of failed reception is sent to the source end through the data exchange control module.

[0045] In this embodiment, at the receiving end, an efficient incremental update logic is designed to parse the received differential data and accurately merge it into the local data set, ensuring the latest status of the data while maintaining the integrity and consistency of the data.

[0046] After data transmission, the sender waits for a confirmation response from the receiver, confirming that the data was received correctly and completely. Furthermore, technologies such as digital signatures and checksums ensure the reliability and traceability of data exchange. If an error is discovered, the problem can be quickly located and the missing or damaged portion can be retransmitted.

[0047] As an improvement to this embodiment, for each type of full data at the source end, the data fingerprint generation module is used to set a priority for each type of full data.

[0048] Correspondingly, the data exchange control module is configured with an adaptive transmission mode, which is used to detect the current network transmission status between the source end and the target end based on the adaptive transmission mode. If the current network transmission status is lower than the threshold, the data exchange control module is used to prioritize the transmission of the encrypted feature vector corresponding to the high-priority full data or the transmission of the encrypted change data, wherein the network transmission status includes the current network bandwidth and latency.

[0049] As a further improvement of this embodiment, a log recording and auditing mechanism is configured in the data exchange control module. Based on the log recording and auditing, the data exchange control module is used to screen the key data of the full data and changed data of the source end, use the screened key data as the data summary, and perform operation log recording on the operations in the data fingerprint generation module, the data exchange control module and the incremental update module to form data operation information, and construct log recording information based on the data summary and the data operation information, wherein the data operation information includes a timestamp and an operation result, and the operation result includes two types: successful reception at the receiving end and failed reception at the receiving end.

[0050] This embodiment maintains a detailed operation log, recording the details of each data exchange, including timestamp, transmission content summary, operation results, etc., to provide a basis for subsequent auditing, troubleshooting and compliance inspection.

[0051] As a further improvement of this embodiment, the data exchange system also includes a compatible interface, which is used to be compatible with components developed by a third party. The components developed by the third party include components for implementing data compression, components for implementing transmission optimization, and components for implementing security enhancement.

[0052] The system of this embodiment provides an API interface and a plug-in mechanism, and third-party developers can develop compatible components, such as new data compression algorithms, transmission optimization strategies, or security enhancement modules, to enhance the flexibility and applicability of the system.

[0053] As a further improvement to this embodiment, the data exchange system also includes a dynamic scaling management module, which is used to support users in adjusting resource allocation of computing nodes where the source and target ends are located, including adding computing nodes, expanding the storage space of each computing node, and optimizing the allocation of network bandwidth between the source and target ends.

[0054] As the amount of data and user demand grow, the system in this embodiment can automatically or manually adjust resource allocation, including adding computing nodes, expanding storage space, or optimizing network bandwidth allocation to ensure continued high performance.

[0055] The development language of the system in this embodiment can adapt to various operating systems, ensuring that the protocol can run seamlessly on multiple operating systems (such as Windows, Linux, macOS) and hardware platforms to meet diverse deployment needs.

[0056] The system is designed following the principle of modularity, allowing easy addition of new data processing modules or replacement of existing modules (such as changing hash algorithms, adding new feature extraction methods) to adapt to changing needs and technological advances.

[0057] The system of this embodiment can be applied to the following scenarios:

[0058] (1) Cloud storage and backup: In cloud storage solutions, this protocol can effectively reduce data redundancy, speed up backup, and ensure the integrity and accuracy of data during recovery;

[0059] (2) Distributed database synchronization: For large-scale distributed database systems, this technical solution can achieve efficient data synchronization, especially between database clusters across countries or regions, significantly reducing network bandwidth consumption;

[0060] (3) Internet of Things (IoT) data transmission: By reducing the amount of data transmission between resource-constrained IoT devices, the battery life of the devices can be extended while maintaining the real-time nature of data updates;

[0061] (4) Content Delivery Network (CDN): Optimize the data distribution strategy of CDN to only push updated or newly added content, improve user access speed, and reduce server bandwidth pressure.

[0062] Example 2:

[0063] The present invention provides a data exchange method based on a data compression identifier, which realizes data exchange between a source end and a target end through the data exchange system disclosed in Example 1. Figure 1 As shown, it includes four steps: source-side data analysis, data fingerprint calculation, data fingerprint transmission, and target-side data update.

[0064] Step S100: Source end data analysis: The data exchange control module is used to analyze the data at the source end, filter out the incremental data and updated local data, and use the incremental data and updated local data as the changed data.

[0065] As a specific implementation of data analysis, incremental data and updated local data are filtered out through Rsync or similar differential algorithms.

[0066] Step S200: Data fingerprint calculation: The user configures multiple hash algorithms and multiple feature vector extraction algorithms in the data fingerprint generation module, performs multi-layer hash calculations on the change data based on the hash algorithm to obtain encrypted change data, and extracts feature vectors from the full data of various data structures through the matching feature vector extraction algorithm. The feature vectors corresponding to the encrypted change data and the full data of various data structures are used as data fingerprints to construct a data fingerprint set.

[0067] As a specific implementation of data fingerprint calculation, users configure three algorithms in the data fingerprint generation module: SHA-256, CRC32, and LSH. For changing data, a two-layer hash algorithm is used for encryption. The first round uses SHA-256 to ensure data integrity and security, while the second round uses the faster hash algorithm CRC32 to reduce the fingerprint code length, balancing security and efficiency. For full-scale structured and semi-structured data, key fields or data patterns are identified as feature vectors, and the fingerprint information is further refined through the combination of these features. For unstructured data, local sensitive hashing (LSH) is used to capture the approximate characteristics of the data as feature vectors.

[0068] Step S300: Data fingerprint transmission: The data exchange control module performs data screening on the data fingerprints in the data fingerprint set, removes duplicate data fingerprints through data screening, encrypts and digitally signs the screened data fingerprint set, and sends the encrypted data fingerprint set to the target end.

[0069] As a specific implementation of data fingerprint transmission, the data fingerprints in the data fingerprint set are screened through ETL tools to reduce the actual transmission volume of subsequent data, and the screened data fingerprint set is encrypted through TLS / SSL encryption method to ensure the security of data during transmission. The encrypted data fingerprint set is sent to the target end and stored to increase the risk defense against data leakage.

[0070] As a specific implementation of data fingerprint screening, data fingerprints in the data fingerprint set are screened through a Bloom filter.

[0071] Step S400: Target end data update: For the encrypted data fingerprint set received by the target end, the encrypted data fingerprint set is decrypted and verified through the incremental update module to obtain a decryption result including the changed data and the corresponding feature vectors of various data structures, and the local data of the target end is updated based on the decryption result. If the update is successful, a successful reception response is sent to the source end through the data exchange control module. If the update fails, a failed reception response is sent to the source end through the data exchange control module.

[0072] In this embodiment, at the receiving end, an efficient incremental update logic is designed to parse the received differential data and accurately merge it into the local data set, ensuring the latest status of the data while maintaining the integrity and consistency of the data.

[0073] After data transmission, the sender waits for a confirmation response from the receiver, confirming that the data was received correctly and completely. Furthermore, technologies such as digital signatures and checksums ensure the reliability and traceability of data exchange. If an error is discovered, the problem can be quickly located and the missing or damaged portion can be retransmitted.

[0074] As an improvement to the method of this embodiment, for each type of full data at the source, a priority is set for each type of full data through the data fingerprint generation module. Accordingly, an adaptive transmission mode is configured in the data exchange control module. When the data fingerprint is transmitted through the ETL tool, the current network transmission status between the source and target ends is detected based on the adaptive transmission mode. If the current network transmission status is below a threshold, the encrypted feature vector corresponding to the high-priority full data or the encrypted change data is preferentially transmitted, where the network transmission status includes the current network bandwidth and latency.

[0075] As a further improvement of this embodiment, a log recording and auditing mechanism is configured in the data exchange control module. Based on the log recording and auditing, key data of the full data and changed data of the source end are screened, and the screened key data is used as the data summary. The operations in the data fingerprint generation module, the data exchange control module and the incremental update module are recorded in the operation log to form data operation information. The log recording information is constructed based on the data summary and the data operation information, wherein the data operation information includes a timestamp and an operation result, and the operation result includes two types: successful reception at the receiving end and failed reception at the receiving end.

[0076] This embodiment maintains a detailed operation log, recording the details of each data exchange, including timestamp, transmission content summary, operation results, etc., to provide a basis for subsequent auditing, troubleshooting and compliance inspection.

[0077] As a further improvement to this embodiment, the user can adjust the resource allocation of the computing nodes where the source and target ends are located, including adding computing nodes, expanding the storage space of each computing node, and optimizing the allocation of network bandwidth between the source and target ends.

[0078] As the amount of data and user demand grows, this embodiment can automatically or manually adjust resource allocation, including adding computing nodes, expanding storage space, or optimizing network bandwidth allocation to ensure continued high performance.

[0079] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.

Claims

1. A data exchange system based on a data compression identifier, characterized in that: Used to implement data exchange between the source and target ends, including plug-in data fingerprint generation module, data exchange control module and incremental update module; The data exchange control module is used to analyze the data at the source end, filter out the incremental data and the updated local data, and use the incremental data and the updated local data as the changed data; The data fingerprint generation module is used to support user configuration of multiple hash algorithms and multiple feature vector extraction algorithms, and is used to perform multi-layer hash calculations on the change data based on the hash algorithm to obtain encrypted change data. For the full data of various data structures, feature vectors are extracted from the full data of each data structure using a matching feature vector extraction algorithm. The feature vectors corresponding to the encrypted change data and the full data of various data structures are used as data fingerprints to construct a data fingerprint set; The data exchange control module is used to screen the data fingerprints in the data fingerprint set, remove duplicate data fingerprints through data screening, encrypt and digitally sign the screened data fingerprint set, and send the encrypted data fingerprint set to the target end; For the encrypted data fingerprint set received by the target end, the incremental update module is used to decrypt and verify the encrypted data fingerprint set, obtain a decryption result including the changed data and the feature vectors corresponding to various data structures, and update the local data of the target end based on the decryption result. If the update is successful, a response of successful reception is sent to the source end through the data exchange control module. If the update fails, a response of failed reception is sent to the source end through the data exchange control module.

2. The data exchange system based on data compression identifier according to claim 1, characterized in that: The data exchange control module is used to filter out incremental data and updated local data through Rsync.

3. The data exchange system based on data compression identifier according to claim 1, characterized in that: For the full amount of structured or semi-structured data, the data fingerprint generation module is used to identify the key fields or data patterns of the data as feature vectors. For unstructured data, the data fingerprint generation module is used to calculate the approximate features of the data as feature vectors through local sensitive hashing.

4. The data exchange system based on data compression identifier according to claim 1, characterized in that: For each type of full data at the source end, the data fingerprint generation module is used to set a priority for each type of full data; The data exchange control module is configured with an adaptive transmission mode, which is used to detect the current network transmission status between the source end and the target end based on the adaptive transmission mode. If the current network transmission status is lower than the threshold, the data exchange control module is used to prioritize the transmission of the encrypted feature vector corresponding to the high-priority full data or the transmission of the encrypted change data, wherein the network transmission status includes the current network bandwidth and latency.

5. The data exchange system based on data compression identifier according to claim 1, characterized in that: The data exchange control module is used to perform data screening on the data fingerprints in the data fingerprint set through a Bloom filter.

6. The data exchange system based on data compression identifier according to claim 1, characterized in that: The data exchange control module is configured with a log recording and auditing mechanism. Based on the log recording and auditing mechanism, the data exchange control module is used to screen the key data of the full data and changed data of the source end, use the screened key data as the data summary, and record the operations in the data fingerprint generation module, the data exchange control module and the incremental update module in the operation log to form data operation information, and construct the log recording information based on the data summary and the data operation information, wherein the data operation information includes a timestamp and an operation result, and the operation result includes two types: successful reception at the receiving end and failed reception at the receiving end.

7. The data exchange system based on data compression identifier according to claim 1, characterized in that: The data exchange control module is configured with an ETL tool, which is used to screen the data fingerprints in the data fingerprint set, encrypt the screened data fingerprint set through the TLS / SSL encryption method, send the encrypted data fingerprint set to the target end, and store the encrypted data fingerprint set.

8. The data exchange system based on data compression identifier according to claim 1, characterized in that: The data exchange system further includes a compatible interface, which is used to be compatible with components developed by a third party. The components developed by the third party include a component for implementing data compression, a component for implementing transmission optimization, and a component for implementing security enhancement.

9. The data exchange system based on data compression identifier according to claim 1, characterized in that: The data exchange system also includes a dynamic scaling management module, which is used to support users in adjusting resource allocation of computing nodes where the source and target ends are located, including adding computing nodes, expanding the storage space of each computing node, and optimizing the allocation of network bandwidth between the source and target ends.

10. A data exchange method based on a data compression identifier, characterized in that: The method for realizing data exchange between a source end and a target end by using a data exchange system based on a data compression identifier according to any one of claims 1 to 9 comprises the following steps: The data exchange control module analyzes the data on the source end, filters out the incremental data and updated local data, and uses the incremental data and updated local data as the changed data; The user configures multiple hash algorithms and multiple feature vector extraction algorithms in the data fingerprint generation module. Based on the hash algorithm, multi-layer hash calculations are performed on the change data to obtain encrypted change data. For the full data of various data structures, feature vectors are extracted from the full data of each data structure using the matching feature vector extraction algorithm. The feature vectors corresponding to the encrypted change data and the full data of various data structures are used as data fingerprints to construct a data fingerprint set. The data exchange control module screens the data fingerprints in the data fingerprint set, removes duplicate data fingerprints through data screening, encrypts and digitally signs the screened data fingerprint set, and sends the encrypted data fingerprint set to the target end; For the encrypted data fingerprint set received by the target end, the encrypted data fingerprint set is decrypted and verified through the incremental update module to obtain the decryption result including the changed data and the corresponding feature vectors of various data structures, and the local data of the target end is updated based on the decryption result. If the update is successful, a response of successful reception is sent to the source end through the data exchange control module. If the update fails, a response of failed reception is sent to the source end through the data exchange control module.

Images