Intelligent data sharing and exchanging system based on synchronization technology
By adopting synchronization technology and AES encryption algorithm in the data sharing and exchange system, combining access control and security audit modules, the data transmission security problem in traditional systems is solved, and efficient encryption and decryption of data transmission and storage is realized, improving the system's Safety and reliability.
Patent Information
- Application Number
- CN202410923807.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-29
AI Technical Summary
Traditional data sharing and exchange systems have security problems such as man-in-the-middle attacks and data tampering during data transmission. The encryption strength of ordinary encryption algorithms is insufficient, difficult to manage keys, and low decryption efficiency, which affects the security and reliability of the system.
Adopt an intelligent data sharing and exchange system based on synchronization technology, including data acquisition, storage, synchronization, exchange, security and other modules. The AES encryption algorithm is used for data encryption and decryption, and combines access control and security audit modules to ensure that data is transmitted and stored. Security.
Through the cooperation of AES encryption algorithm and access control module, the powerful encryption capability of data during transmission and decryption capability before storage is achieved, ensuring the security and reliability of data, protecting data from interception and leakage, and satisfying the System exchange with different security requirements.
Smart Images

Figure CN119051884A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of data sharing and exchange, and in particular relates to an intelligent data sharing and exchange system based on synchronization technology. Background Art
[0002] The data sharing and exchange system is an information system used to realize data sharing and exchange between different systems and organizations. It realizes seamless connection and circulation of data by providing unified data interface, data format and data transmission protocol, thereby breaking the information island and improving the efficiency of data utilization. The main components of the data sharing and exchange system include: Data source: The data sharing and exchange system needs to connect to various data sources, including various databases, file systems, service interfaces, etc., to obtain data that needs to be shared and exchanged. Data interface: The data sharing and exchange system provides a unified data interface and supports the input and output of various data formats to facilitate data exchange between different systems. Data transmission: The data sharing and exchange system adopts efficient and secure data transmission protocols, such as HTTP, FTP, WebService, etc., to realize rapid data transmission and exchange. Data processing: The data sharing and exchange system can clean, convert, encrypt and other processes on the data to ensure the quality and security of the data. Data management: The data sharing and exchange system provides data management functions, including data query, data statistics, data authority management, etc., to effectively manage the shared and exchanged data.
[0003] However, traditional data sharing and exchange systems may have problems such as man-in-the-middle attacks and data tampering during data transmission, and ordinary encryption algorithms may have problems such as insufficient encryption strength, difficult key management, and low decryption efficiency, which in turn affects the security and reliability of the entire system. Summary of the invention
[0004] The purpose of the present invention is to provide an intelligent data sharing and exchange system based on synchronization technology in order to solve the above-mentioned problems.
[0005] The technical solution adopted by the present invention is as follows: an intelligent data sharing and exchange system based on synchronization technology, characterized in that: the system includes: a data acquisition module, a data storage module, a data exchange module, a data synchronization module, a user interface module, a system management module and a data security module;
[0006] After the data acquisition module collects data from various data sources, it transmits the cleaned and formatted data to the data storage module;
[0007] The data storage module receives the data from the data acquisition module and stores it in a central database or a data warehouse;
[0008] The data synchronization module monitors data changes in the data storage module and ensures that these changes are synchronized to other nodes or replicas in the system;
[0009] The data storage module provides the necessary data access interface for the data synchronization module;
[0010] After the data is updated, the data synchronization module notifies the data exchange module to perform data mapping and conversion;
[0011] The data exchange module is updated according to the data provided by the synchronization module and performs data exchange according to a predetermined exchange protocol;
[0012] During the data exchange process, the data exchange module uses the encryption and access control mechanism provided by the data security module to protect the data security module from monitoring the activities of the data exchange module to ensure that the data exchange complies with the security policy;
[0013] The user interface module interacts with the data security module to verify the user's access rights when the user requests to access data; the user interface module allows the system administrator to configure and manage the system through a Web management interface or a command line tool;
[0014] The system management module receives configuration change requests from the user interface module and updates the system configuration accordingly;
[0015] The data security module is internally provided with a data encryption module, a data decryption module, an access control module and a security audit module;
[0016] The data encryption module performs encryption when the data exchange module is running. This module is responsible for sending data to other systems or external users. It performs encryption before the data leaves the system to protect the security of the data during transmission and prevent the data from being intercepted or leaked during transmission.
[0017] The reason why the data decryption module performs decryption when the data synchronization module is running is that this module is responsible for receiving data from other nodes or systems and synchronizing it to local storage; decryption is performed after data synchronization and before storage to ensure that the stored data is available while protecting the security of the data in a static state.
[0018] In a preferred embodiment, the data acquisition module is the entrance of the intelligent data sharing and exchange system, responsible for obtaining raw data from various data sources; the data source identification component automatically detects and accesses different types of data sources, including relational databases, non-relational databases, file systems, and API interfaces; the data extraction component extracts the required data from these data sources according to predetermined rules and strategies; the data cleaning component formats the extracted data, cleans out invalid, erroneous or duplicated data, and ensures the accuracy and quality of the data; after this series of processing, the data can be passed to the next stage - the data storage module;
[0019] The data storage module is responsible for storing and managing the data received from the data acquisition module; the central database is used to store all collected data and provide persistent storage for the system; the data warehouse classifies and integrates the data, builds a model suitable for data analysis, and facilitates data analysis and use; the cache mechanism temporarily stores frequently accessed data, improves the system response speed, and reduces the access pressure on the central database; the design of the data storage module needs to take into account the data capacity, performance, reliability and scalability to meet the needs of different business scenarios.
[0020] In a preferred embodiment, the data synchronization module ensures data consistency between nodes in the system; the synchronization algorithm is used to achieve data consistency to ensure that data updates of each node remain synchronized; the conflict resolution mechanism handles data conflict problems in concurrent operations to ensure data consistency; data verification verifies data before and after synchronization to ensure data integrity and accuracy; the design of the data synchronization module needs to take into account data consistency, system availability and partition fault tolerance to ensure that the system can operate normally under various circumstances.
[0021] In a preferred embodiment, the data exchange module is responsible for realizing data exchange between the system and the external system; the exchange protocol defines the format and rules of data exchange, ensuring the standardization and universality of data exchange; the exchange engine executes the data exchange logic, realizes data mapping and conversion, and converts data from one format to another to meet the needs of different systems; the API interface provides a standard data access interface for external system calls to realize data sharing and exchange; the design of the data exchange module needs to take into account the universality, ease of use and scalability of the data to meet the ever-changing business needs.
[0022] In a preferred embodiment, the encryption process of the data encryption module is:
[0023] Choose a key length (128, 192, or 256 bits), denoted as k;
[0024] Generate a random initialization vector (IV), denoted as IV, whose length is the same as the AES block size (128 bits);
[0025] Split the plaintext data PP into blocks of 128 bits in length, denoted as P1, P2, ..., Pn;
[0026] Perform an XOR operation on each plaintext block Pi and the previous ciphertext block Ci-1 or IV, and then apply the AES encryption algorithm to obtain the ciphertext block Ci;
[0027] The encryption formula is:
[0028] C1=AES Encrypt (P1⊕IV,k)
[0029] Ci=AES Encrypt (Pi⊕Ci-1,k)for i=2,...,n;
[0030] Among them: AES_Encrypt: is a function that represents the AES encryption operation; it accepts a plaintext block P_iPi (128 bits) and a key kk (128, 192 or 256 bits) and outputs a ciphertext block C_iCi (128 bits);
[0031] AES_Decrypt: is a function that represents the AES decryption operation; it accepts a ciphertext block C_iCi (128 bits) and a key kk (128, 192, or 256 bits) and outputs a plaintext block P_iPi (128 bits);
[0032] ⊕ is the XOR operator; when performing an XOR operation on two binary sequences of equal length, the result is 0 if the bits in the same position are the same, and 1 if they are different;
[0033] IV stands for Initialization Vector, which is a randomly generated bit sequence that is XORed with the first plaintext block to ensure that even if the same plaintext is encrypted multiple times, different ciphertexts will be generated. In CBC mode, the length of IV is the same as the block size of AES, which is 128 bits.
[0034] Pi represents the i-th block in the plaintext data;
[0035] Ci represents the i-th block in the ciphertext data;
[0036] Pi′ represents the intermediate plaintext block in the decryption process, that is, the plaintext block obtained by decryption before the XOR operation; n represents the total number of plaintext or ciphertext blocks.
[0037] In a preferred embodiment, the decryption process of the data decryption module is as follows:
[0038] Use the same key k and IV used when encrypting;
[0039] Apply the AES decryption algorithm to each ciphertext block Ci to obtain the intermediate plaintext block Pi′;
[0040] XOR Pi′ with the previous ciphertext block Ci-1 or IV to obtain the original ciphertext block Pi;
[0041] The decryption formula is:
[0042] P1′=AES Decrypt (C1,k)
[0043] Pi=Pi′⊕Ci-1for i=1,...,n-1
[0044] Pn=Pn′⊕IV
[0045] Among them: AES_Encrypt: is a function that represents the AES encryption operation; it accepts a plaintext block P_iPi (128 bits) and a key kk (128, 192 or 256 bits) and outputs a ciphertext block C_iCi (128 bits);
[0046] AES_Decrypt: is a function that represents the AES decryption operation; it accepts a ciphertext block C_iCi (128 bits) and a key kk (128, 192, or 256 bits) and outputs a plaintext block P_iPi (128 bits);
[0047] ⊕ is the XOR operator; when performing an XOR operation on two binary sequences of equal length, the result is 0 if the bits in the same position are the same, and 1 if they are different;
[0048] IV stands for Initialization Vector, which is a randomly generated bit sequence that is XORed with the first plaintext block to ensure that even if the same plaintext is encrypted multiple times, different ciphertexts will be generated. In CBC mode, the length of IV is the same as the block size of AES, which is 128 bits.
[0049] Pi represents the i-th block in the plaintext data;
[0050] Ci represents the i-th block in the ciphertext data;
[0051] Pi′ represents the intermediate plaintext block in the decryption process, that is, the plaintext block obtained by decryption before the XOR operation; n represents the total number of plaintext or ciphertext blocks.
[0052] In a preferred embodiment, the workflow of the access control module includes:
[0053] User authentication: Before a user attempts to access the system, the access control module requires the user to provide proof of identity, usually through a username and password, or a more secure two-factor authentication method;
[0054] User authorization: Once the user is authenticated, the access control module checks whether the user has permission to perform the requested action; this usually involves role-based access control (RBAC), where the user's role determines the resources they can access and the actions they can perform on those resources;
[0055] Permission management: System administrators configure and manage user permissions, including read, write, modify, or delete permissions for specific data items; these permissions are broken down to individual data records or specific data sets;
[0056] Access Control List (ACL): The access control module uses ACL to define which users or user groups are authorized to access which resources; ACL is statically configured and also dynamically generated based on the user's dynamic attributes;
[0057] Operational Auditing: The access control module records all access attempts, including successful and failed attempts, to facilitate subsequent auditing and analysis.
[0058] In a preferred embodiment, the security audit module is internally provided with:
[0059] Behavior monitoring module: The security audit module monitors the system's data access and operation behavior in real time, including data reading, writing, modifying and deleting operations;
[0060] Logging module: The module will record all data operation behaviors in the log, including the time of the operation, the user, the type of operation performed, and the detailed information of the data objects involved in the operation;
[0061] Anomaly detection module: By analyzing the logs, the security audit module detects abnormal behavior patterns;
[0062] Audit Analysis Module: Auditors use the tools provided by the Security Audit Module to analyze logs to identify potential security threats or violations;
[0063] Compliance Check Module: The security audit module also helps organizations meet the requirements of regulations and standards;
[0064] Report generation module: The module can generate regular security reports, summarizing trends in data access and operations, as well as potential security risks, to provide decision support for management.
[0065] In a preferred embodiment, the user interface module is the interface for users to interact with the intelligent data sharing and exchange system; the Web management interface allows users to manage data and system settings through a browser, providing an intuitive and easy-to-use operation interface; the command line tool provides a command line operation interface for advanced users, allowing them to manage data through scripts and automation tools; the mobile application supports access to and operation of data through mobile devices, allowing users to manage data anytime and anywhere; the design of the user interface module needs to take into account the user's convenience, ease of use and personalized needs to improve the user experience.
[0066] In a preferred embodiment, the system management module is the control center of the intelligent data sharing and exchange system, responsible for managing system configuration information and parameters; configuration management allows system administrators to manage and update system configuration information and parameters to adapt to changing business needs; monitoring and alarm monitors the system operation status, and promptly issues alarms when anomalies are found, helping administrators to promptly discover and solve problems; log management records system operation logs for problem tracking and performance analysis, helping administrators optimize system performance; the design of the system management module needs to take into account the stability, maintainability and scalability of the system to ensure long-term stable operation of the system.
[0067] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0068] 1. In the present invention, the data encryption module performs encryption when the data exchange module is running. This module is responsible for sending data to other systems or external users. Encrypting data before it leaves the system can protect the security of the data during transmission and prevent the data from being intercepted or leaked during transmission. By using the AES encryption algorithm, the data encryption module can provide powerful encryption capabilities to ensure the security of data during transmission. The data encryption module supports multiple encryption standards, so that the system can exchange data with systems with different security requirements. The encryption process of the data encryption module includes steps such as selecting a key length, generating an initialization vector, segmenting plaintext data, and applying the AES encryption algorithm, which ensures the security and reliability of the data encryption process. The data decryption module performs decryption when the data synchronization module is running. This module is responsible for receiving data from other nodes or systems and synchronizing it to local storage. Decryption after data synchronization and before storage can ensure that the stored data is available and protect the security of the data in a static state. By using the AES decryption algorithm, the data decryption module can provide powerful decryption capabilities to ensure the security of the data before storage. The decryption process of the data decryption module includes using the same key and IV used during encryption, applying the AES decryption algorithm, and XOR operations, which ensures the security and reliability of the data decryption process.
[0069] 2. In the present invention, the access control module ensures that only appropriate users can access specific data through user identity authentication and authorization, thereby protecting personal privacy and sensitive information. The access control module supports role-based access control (RBAC), allowing system administrators to flexibly configure and manage user permissions. The access control module uses access control lists (ACLs) to define which users or user groups are authorized to access which resources, thereby improving the security of the system. The access control module records all access attempts, including successful and failed attempts, to facilitate subsequent audits and analysis, thereby improving the maintainability of the system. The security audit module monitors the data access and operation behaviors of the system in real time, including operations such as reading, writing, modifying and deleting data, thereby improving the security of the system. The security audit module records all data operation behaviors, including detailed information such as the time of the operation, the user, the type of operation performed, and the data objects involved in the operation, thereby providing a basis for auditing and analysis. The security audit module helps organizations meet the requirements of regulations and standards, such as GDPR, HIPAA, etc., which usually require detailed recording and review of data access, thereby improving the compliance of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0070] Figure 1 is the overall system block diagram of the present invention;
[0071] Figure 2 This is a system block diagram of the data security module in the present invention. DETAILED DESCRIPTION
[0072] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0073] Reference Figure 1-2 ,
[0074] Example:
[0075] An intelligent data sharing and exchange system based on synchronization technology, the system includes: a data acquisition module, a data storage module, a data exchange module, a data synchronization module, a user interface module, a system management module and a data security module;
[0076] After the data acquisition module collects data from various data sources, it transmits the cleaned and formatted data to the data storage module.
[0077] The data storage module receives the data from the data acquisition module and stores it in a central database or data warehouse.
[0078] The data synchronization module monitors data changes in the data storage module and ensures that these changes are synchronized to other nodes or replicas in the system.
[0079] The data storage module provides the necessary data access interface for the data synchronization module.
[0080] After the data is updated, the data synchronization module notifies the data exchange module to map and convert the data.
[0081] The data exchange module is updated according to the data provided by the synchronization module and performs data exchange according to a predetermined exchange protocol.
[0082] During the data exchange process, the data exchange module uses the encryption and access control mechanisms provided by the data security module to protect the data security module and monitor the activities of the data exchange module to ensure that the data exchange complies with the security policy.
[0083] The user interface module interacts with the data security module to verify the user's access rights when the user requests to access data. The user interface module allows system administrators to configure and manage the system through a web management interface or command line tools.
[0084] The system management module receives configuration change requests from the user interface module and updates the system configuration accordingly.
[0085] The data security module is internally provided with a data encryption module, a data decryption module, an access control module and a security audit module;
[0086] The data encryption module encrypts data when the data exchange module is running. This module is responsible for sending data to other systems or external users. Encrypting data before it leaves the system can protect the security of data during transmission and prevent data from being intercepted or leaked during transmission.
[0087] The reason why the data decryption module performs decryption when the data synchronization module is running is that this module is responsible for receiving data from other nodes or systems and synchronizing it to local storage. Decryption after data synchronization and before storage can ensure that the stored data is available and protect the security of the data in a static state.
[0088] The data acquisition module is the entrance to the intelligent data sharing and exchange system, responsible for obtaining raw data from various data sources. The data source identification component automatically detects and accesses different types of data sources, such as relational databases, non-relational databases, file systems, API interfaces, etc. The data extraction component extracts the required data from these data sources according to predetermined rules and strategies. The data cleaning component formats the extracted data, cleans up invalid, erroneous or duplicate data, and ensures the accuracy and quality of the data. After this series of processing, the data can be passed to the next stage - the data storage module;
[0089] The data storage module is responsible for storing and managing the data received from the data acquisition module. The central database is used to store all collected data and provide persistent storage for the system. The data warehouse classifies and integrates the data, builds a model suitable for data analysis, and facilitates data analysis and use. The cache mechanism temporarily stores frequently accessed data to improve system response speed and reduce access pressure on the central database. The design of the data storage module needs to take into account the capacity, performance, reliability and scalability of the data to meet the needs of different business scenarios.
[0090] The data synchronization module ensures data consistency between nodes in the system. The synchronization algorithm is used to achieve data consistency and ensure that data updates of each node are synchronized. The conflict resolution mechanism handles data conflicts in concurrent operations to ensure data consistency. Data verification verifies data before and after synchronization to ensure data integrity and accuracy. The design of the data synchronization module needs to take into account data consistency, system availability, and partition fault tolerance to ensure that the system can operate normally under various circumstances.
[0091] The data exchange module is responsible for realizing data exchange between the system and external systems. The exchange protocol defines the format and rules of data exchange, ensuring the standardization and universality of data exchange. The exchange engine executes the data exchange logic, realizes data mapping and conversion, and converts data from one format to another to meet the needs of different systems. The API interface provides a standard data access interface for external system calls to realize data sharing and exchange. The design of the data exchange module needs to take into account the universality, ease of use and scalability of data to meet the ever-changing business needs.
[0092] The encryption process of the data encryption module is:
[0093] Choose a key length (128, 192, or 256 bits), denoted by k.
[0094] Generate a random initialization vector (IV), denoted as IV, whose length is the same as the block size of AES (128 bits).
[0095] The plaintext data PP is divided into blocks of 128 bits in length, denoted as P1, P2, ..., Pn.
[0096] Perform an XOR operation on each plaintext block Pi and the previous ciphertext block Ci-1 or IV, and then apply the AES encryption algorithm to obtain the ciphertext block Ci;
[0097] The encryption formula is:
[0098] C1=AES Encrypt (P1⊕IV,k)
[0099] Ci=AES Encrypt (Pi⊕Ci-1,k)for i=2,...,n;
[0100] Among them: AES_Encrypt: is a function that represents the AES encryption operation. It accepts a plaintext block P_iPi (128 bits) and a key kk (128, 192 or 256 bits) and outputs a ciphertext block C_iCi (128 bits).
[0101] AES_Decrypt: is a function that represents the AES decryption operation. It accepts a ciphertext block C_iCi (128 bits) and a key kk (128, 192, or 256 bits) and outputs a plaintext block P_iPi (128 bits).
[0102] ⊕ is the XOR operator. When performing an XOR operation on two binary sequences of equal length, the result is 0 if the bits in the same position are the same, and 1 if they are different.
[0103] IV stands for Initialization Vector, which is a randomly generated bit sequence that is XORed with the first plaintext block to ensure that even if the same plaintext is encrypted multiple times, different ciphertexts will be generated. In CBC mode, the length of IV is the same as the block size of AES, which is 128 bits.
[0104] Pi represents the i-th block in the plaintext data.
[0105] Ci represents the i-th block in the ciphertext data.
[0106] Pi′ represents the intermediate plaintext block in the decryption process, that is, the plaintext block obtained by decryption before the XOR operation.
[0107] n represents the total number of plaintext or ciphertext blocks.
[0108] The decryption process of the data decryption module is:
[0109] Use the same key k and IV that you used when encrypting.
[0110] Apply the AES decryption algorithm to each ciphertext block Ci to obtain the intermediate plaintext block Pi′.
[0111] XOR Pi′ with the previous ciphertext block Ci-1 or IV to obtain the original ciphertext block Pi;
[0112] The decryption formula is:
[0113] P1′=AES Decrypt (C1,k)
[0114] Pi=Pi′⊕Ci-1for i=1,...,n-1
[0115] Pn=Pn′⊕IV
[0116] Among them: AES_Encrypt: is a function that represents the AES encryption operation. It accepts a plaintext block P_iPi (128 bits) and a key kk (128, 192 or 256 bits) and outputs a ciphertext block C_iCi (128 bits).
[0117] AES_Decrypt: is a function that represents the AES decryption operation. It accepts a ciphertext block C_iCi (128 bits) and a key kk (128, 192, or 256 bits) and outputs a plaintext block P_iPi (128 bits).
[0118] ⊕ is the XOR operator. When performing an XOR operation on two binary sequences of equal length, the result is 0 if the bits in the same position are the same, and 1 if they are different.
[0119] IV stands for Initialization Vector, which is a randomly generated bit sequence that is XORed with the first plaintext block to ensure that even if the same plaintext is encrypted multiple times, different ciphertexts will be generated. In CBC mode, the length of IV is the same as the block size of AES, which is 128 bits.
[0120] Pi represents the i-th block in the plaintext data.
[0121] Ci represents the i-th block in the ciphertext data.
[0122] Pi′ represents the intermediate plaintext block in the decryption process, that is, the plaintext block obtained by decryption before the XOR operation.
[0123] n represents the total number of plaintext or ciphertext blocks.
[0124] The workflow of the access control module includes:
[0125] User authentication: Before a user attempts to access the system, the access control module requires the user to provide proof of identity, usually through a username and password, or a more secure two-factor authentication method.
[0126] User authorization: Once the user is authenticated, the access control module checks whether the user has the permissions to perform the requested operation. This usually involves role-based access control (RBAC), where the user's role determines the resources they can access and the operations they can perform on those resources.
[0127] Permission management: System administrators can configure and manage user permissions, including read, write, modify, or delete permissions for specific data items. These permissions can be refined to individual data records or specific data sets.
[0128] Access Control List (ACL): The access control module uses ACL to define which users or user groups are authorized to access which resources. ACL can be statically configured or dynamically generated based on the user's dynamic attributes (such as department, position, etc.).
[0129] Operational Auditing: The access control module records all access attempts, including successful and failed attempts, to facilitate subsequent auditing and analysis.
[0130] The internal settings of the security audit module are:
[0131] Behavior monitoring module: The security audit module monitors the system's data access and operation behavior in real time, including data reading, writing, modifying and deleting operations.
[0132] Logging module: The module will record all data operation behaviors in the log, including the time of the operation, the user, the type of operation performed, the data objects involved in the operation, and other detailed information.
[0133] Anomaly detection module: By analyzing logs, the security audit module can detect abnormal behavior patterns, such as frequent failed access attempts, abnormal access times, or a surge in data access, thereby triggering alarms.
[0134] Audit Analysis Module: Auditors can use the tools provided by the Security Audit Module to analyze logs to identify potential security threats or violations.
[0135] Compliance Check Module: The security audit module can also help organizations meet the requirements of regulations and standards, such as GDPR, HIPAA, etc., which often require detailed logging and review of data access.
[0136] Report generation module: The module can generate regular security reports, summarizing trends in data access and operations, as well as potential security risks, to provide decision support for management.
[0137] The user interface module is the interface for users to interact with the intelligent data sharing and exchange system. The web management interface allows users to manage data and system settings through a browser, providing an intuitive and easy-to-use operation interface. The command line tool provides a command line operation interface for advanced users, allowing them to manage data through scripts and automation tools. Mobile applications support access and operation of data through mobile devices, allowing users to manage data anytime, anywhere. The design of the user interface module needs to take into account user convenience, ease of use, and personalized needs to improve the user experience.
[0138] The system management module is the control center of the intelligent data sharing and exchange system, responsible for managing system configuration information and parameters. Configuration management allows system administrators to manage and update system configuration information and parameters to adapt to changing business needs. Monitoring and alarming monitor the system operation status, and timely alarm when abnormalities are found, helping administrators to find and solve problems in a timely manner. Log management records system operation logs for problem tracking and performance analysis, helping administrators optimize system performance. The design of the system management module needs to take into account the stability, maintainability and scalability of the system to ensure long-term stable operation of the system.
[0139] In the present invention, the data encryption module performs encryption when the data exchange module is running, and this module is responsible for sending data to other systems or external users. Encrypting data before leaving the system can protect the security of data during transmission and prevent data from being intercepted or leaked during transmission. By using the AES encryption algorithm, the data encryption module can provide powerful encryption capabilities to ensure the security of data during transmission. The data encryption module supports multiple encryption standards, so that the system can exchange data with systems with different security requirements. The encryption process of the data encryption module includes steps such as selecting a key length, generating an initialization vector, segmenting plaintext data, and applying the AES encryption algorithm, which ensures the security and reliability of the data encryption process. The data decryption module performs decryption when the data synchronization module is running, and this module is responsible for receiving data from other nodes or systems and synchronizing it to local storage. Decryption is performed after data synchronization and before storage, which can ensure that the stored data is available and protect the security of the data in a static state. By using the AES decryption algorithm, the data decryption module can provide powerful decryption capabilities to ensure the security of data before storage. The decryption process of the data decryption module includes using the same key and IV used during encryption, applying the AES decryption algorithm, and XOR operations, which ensures the security and reliability of the data decryption process.
[0140] In the present invention, the access control module ensures that only appropriate users can access specific data through user identity authentication and authorization, thereby protecting personal privacy and sensitive information. The access control module supports role-based access control (RBAC), allowing system administrators to flexibly configure and manage user permissions. The access control module uses access control lists (ACLs) to define which users or user groups are authorized to access which resources, thereby improving the security of the system. The access control module records all access attempts, including successful and failed attempts, to facilitate subsequent audits and analysis, thereby improving the maintainability of the system. The security audit module monitors the data access and operation behaviors of the system in real time, including operations such as reading, writing, modifying and deleting data, thereby improving the security of the system. The security audit module records all data operation behaviors, including detailed information such as the time of the operation, the user, the type of operation performed, and the data objects involved in the operation, thereby providing a basis for auditing and analysis. The security audit module helps organizations meet the requirements of regulations and standards, such as GDPR, HIPAA, etc., which usually require detailed recording and review of data access, thereby improving the compliance of the system.
[0141] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0142] The above description enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An intelligent data sharing and exchange system based on synchronization technology, characterized in that: The system includes: a data acquisition module, a data storage module, a data exchange module, a data synchronization module, a user interface module, a system management module and a data security module; After the data acquisition module collects data from various data sources, it transmits the cleaned and formatted data to the data storage module; The data storage module receives the data from the data acquisition module and stores it in a central database or a data warehouse; The data synchronization module monitors data changes in the data storage module and ensures that these changes are synchronized to other nodes or replicas in the system; The data storage module provides the necessary data access interface for the data synchronization module; After the data is updated, the data synchronization module notifies the data exchange module to perform data mapping and conversion; The data exchange module is updated according to the data provided by the synchronization module and performs data exchange according to a predetermined exchange protocol; During the data exchange process, the data exchange module uses the encryption and access control mechanism provided by the data security module to protect the data security module from monitoring the activities of the data exchange module to ensure that the data exchange complies with the security policy; The user interface module interacts with the data security module to verify the user's access rights when the user requests to access data; the user interface module allows the system administrator to configure and manage the system through a Web management interface or a command line tool; The system management module receives configuration change requests from the user interface module and updates the system configuration accordingly; The data security module is internally provided with a data encryption module, a data decryption module, an access control module and a security audit module; The data encryption module performs encryption when the data exchange module is running, and the data decryption module performs decryption when the data synchronization module is running.
2. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The data acquisition module is the entrance to the intelligent data sharing and exchange system, responsible for obtaining raw data from various data sources; the data source identification component automatically detects and accesses different types of data sources, including relational databases, non-relational databases, file systems, and API interfaces; the data extraction component extracts the required data from these data sources according to predetermined rules and strategies; the data cleaning component formats the extracted data, cleans out invalid, erroneous or duplicated data, and ensures the accuracy and quality of the data; after this series of processing, the data can be passed to the next stage - the data storage module; The data storage module is responsible for storing and managing the data received from the data acquisition module; the central database is used to store all collected data and provide persistent storage for the system; the data warehouse classifies and integrates the data, builds a model suitable for data analysis, and facilitates data analysis and use; the cache mechanism temporarily stores frequently accessed data, improves the system response speed, and reduces the access pressure on the central database; the design of the data storage module needs to take into account the data capacity, performance, reliability and scalability to meet the needs of different business scenarios.
3. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The data synchronization module ensures data consistency between nodes in the system; the synchronization algorithm is used to achieve data consistency and ensure that data updates of each node remain synchronized; the conflict resolution mechanism handles data conflicts in concurrent operations to ensure data consistency; Data verification verifies data before and after synchronization to ensure data integrity and accuracy; the design of the data synchronization module needs to take into account data consistency, system availability and partition fault tolerance to ensure that the system can operate normally under various circumstances.
4. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The data exchange module is responsible for realizing data exchange between the system and the external system; the exchange protocol defines the format and rules of data exchange.
5. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The encryption process of the data encryption module is: Choose a key length (128, 192, or 256 bits), denoted as k; Generate a random initialization vector (IV), denoted as IV, whose length is the same as the AES block size (128 bits); Split the plaintext data PP into blocks of 128 bits in length, denoted as P1, P2, ..., Pn; Perform an XOR operation on each plaintext block Pi and the previous ciphertext block Ci-1 or IV, and then apply the AES encryption algorithm to obtain the ciphertext block Ci; The encryption formula is: C1=AES Encrypt (P1⊕IV,k) Ci=AES Encrypt (Pi⊕Ci-1,k)for i=2,...,n; Among them: AES_Encrypt: is a function that represents the AES encryption operation; it accepts a plaintext block P_iPi (128 bits) and a key kk (128, 192 or 256 bits) and outputs a ciphertext block C_iCi (128 bits); AES_Decrypt: is a function that represents the AES decryption operation; it accepts a ciphertext block C_iCi (128 bits) and a key kk (128, 192, or 256 bits) and outputs a plaintext block P_iPi (128 bits); ⊕ is the XOR operator; when performing an XOR operation on two binary sequences of equal length, the result is 0 if the bits in the same position are the same, and 1 if they are different; IV stands for Initialization Vector, which is a randomly generated bit sequence that is XORed with the first plaintext block to ensure that even if the same plaintext is encrypted multiple times, different ciphertexts will be generated. In CBC mode, the length of IV is the same as the block size of AES, which is 128 bits. Pi represents the i-th block in the plaintext data; Ci represents the i-th block in the ciphertext data; Pi′ represents the intermediate plaintext block in the decryption process, that is, the plaintext block obtained by decryption before the XOR operation; n represents the total number of plaintext or ciphertext blocks.
6. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The decryption process of the data decryption module uses the same key k and IV used during encryption; Apply the AES decryption algorithm to each ciphertext block Ci to obtain the intermediate plaintext block Pi′; XOR Pi′ with the previous ciphertext block Ci-1 or IV to obtain the original ciphertext block Pi; The decryption formula is: P1′=AES Decrypt (C1,k) Pi=Pi′⊕Ci-1for i=1,...,n-1 Pn=Pn′⊕IV Among them: AES_Encrypt: is a function that represents the AES encryption operation; it accepts a plaintext block P_iPi (128 bits) and a key kk (128, 192 or 256 bits) and outputs a ciphertext block C_iCi (128 bits); AES_Decrypt: is a function that represents the AES decryption operation; it accepts a ciphertext block C_iCi (128 bits) and a key kk (128, 192, or 256 bits) and outputs a plaintext block P_iPi (128 bits); ⊕ is the XOR operator; when performing an XOR operation on two binary sequences of equal length, the result is 0 if the bits in the same position are the same, and 1 if they are different; IV stands for Initialization Vector, which is a randomly generated bit sequence that is XORed with the first plaintext block to ensure that even if the same plaintext is encrypted multiple times, different ciphertexts will be generated. In CBC mode, the length of IV is the same as the block size of AES, which is 128 bits. Pi represents the i-th block in the plaintext data; Ci represents the i-th block in the ciphertext data; Pi′ represents the intermediate plaintext block in the decryption process, that is, the plaintext block obtained by decryption before the XOR operation; n represents the total number of plaintext or ciphertext blocks.
7. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The workflow of the access control module includes: User authentication: Before a user attempts to access the system, the access control module requires the user to provide proof of identity, usually through a username and password, or a more secure two-factor authentication method; User authorization: Once the user is authenticated, the access control module checks whether the user has permission to perform the requested action; this usually involves role-based access control, where the user's role determines the resources they can access and the actions they can perform on those resources; Permission management: System administrators configure and manage user permissions, including read, write, modify, or delete permissions for specific data items; these permissions are broken down to individual data records or specific data sets; Access Control List: The access control module uses ACL to define which users or user groups are authorized to access which resources; Operational Auditing: The access control module records all access attempts, including successful and failed attempts.
8. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The internal configuration of the security audit module includes: Behavior monitoring module: The security audit module monitors the system's data access and operation behavior in real time, including data reading, writing, modifying and deleting operations; Logging module: The module will record all data operation behaviors in the log, including the time of the operation, the user, the type of operation performed, and the detailed information of the data objects involved in the operation; Anomaly detection module: By analyzing the logs, the security audit module detects abnormal behavior patterns; Audit Analysis Module: Auditors use the tools provided by the Security Audit Module to analyze logs to identify potential security threats or violations.
9. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The user interface module is the interface for users to interact with the intelligent data sharing and exchange system; the Web management interface allows users to manage data and system settings through a browser, and the command line tool provides a command line operation interface for advanced users, allowing data to be managed through scripts and automation tools.
10. The intelligent data sharing and exchange system based on synchronization technology as claimed in claim 1, characterized in that: The system management module is the control center of the intelligent data sharing and exchange system, responsible for managing system configuration information and parameters; configuration management allows system administrators to manage and update system configuration information and parameters.
Citation Information
Cited By
Data docking system based on API
CN119830325A
