Data exchange method and device

By negotiating a contract agreement between the data provider and the data consumer, establishing an instance connection, and using technologies such as NiFi instance and certificate verification, the problem of insecure data transmission is solved and secure and reliable data exchange is achieved.

CN120658436APending Publication Date: 2025-09-16SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510744071.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-05
Publication Date
2025-09-16

AI Technical Summary

Technical Problem

Data cannot be transmitted securely and reliably during the transmission and exchange process, and the data provider and consumer need to establish a transmission relationship to achieve intercommunication.

Method used

The contract offer is created by the data provider, the data consumer proposes modification opinions, the two parties negotiate and determine the contract agreement, build an instance connection, ensure the security and reliability of data exchange, use the NiFi instance for data transmission, and ensure data integrity through certificate verification and compression algorithm.

Benefits of technology

The invention realizes safe and reliable data transmission between the data providing device and the data consuming device, reduces legal disputes, and improves the security and reliability of data transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120658436A_ABST
    Figure CN120658436A_ABST
Patent Text Reader

Abstract

The invention provides a data exchange method and device, and the method comprises the steps: a data providing device creates a contract offer which represents that the data providing device proposes a proposal with constraining force, a data consumption device obtains an asset directory sent by the data providing device, proposes a modification opinion based on the contract offer, and transmits the modification opinion to the data providing device; the data providing device negotiates with the data consumption device and determines a contract agreement based on the modification suggestion, the data providing device constructs a first instance, the data consumption device constructs a second instance, the first instance is in communication connection with the second instance, and the data consumption device constructs the contract agreement based on the contract agreement. Data provided by the data providing device is obtained through the first instance, and the data providing device provides the data to the data consumption device through the second instance based on the contract agreement. The contract agreement is constructed through the data providing device and the data consumption device, and safe and reliable data transmission between the data providing device and the data consumption device is ensured based on the contract agreement.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data circulation, and in particular to a data exchange method and device. Background Art

[0002] As the world rapidly enters the digital economy, data has become a key production factor driving economic development in various countries, and its economic and strategic value is becoming increasingly prominent. Cross-border data flow has become a crucial hub for maintaining global economic activity and an inevitable trend in global economic development. Currently, data transmission and exchange cannot be carried out securely and reliably. Furthermore, each data transfer between data providers and data consumers requires the establishment of a transmission relationship to achieve data interoperability. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention is proposed. The embodiments of the present invention provide a data exchange method and device, which solve the problem that data cannot be transmitted safely and reliably.

[0004] According to one aspect of the present invention, there is provided a data exchange method, comprising:

[0005] The data providing device creates a contract offer; wherein the contract offer represents a binding proposal made by the data providing device;

[0006] The data consumption device obtains the asset catalog sent by the data providing device and proposes modification suggestions based on the contract offer;

[0007] The data providing device negotiates with the data consuming device based on the modification suggestions and determines a contract agreement;

[0008] The data providing device constructs a first instance;

[0009] The data consumption device establishes a second instance; wherein the first instance and the second instance are communicatively connected;

[0010] The data consumption device obtains the data provided by the data providing device through the first instance based on the contract agreement;

[0011] The data providing device provides data to the data consuming device through the second instance based on the contract agreement.

[0012] In one embodiment, the data providing device providing data to the data consuming device through the second instance based on the contract agreement includes:

[0013] The data consumption device obtains the provider certificate sent by the data providing device; wherein the provider certificate is used to authorize the data providing device to access;

[0014] The data consumption device imports the provider certificate into a trust store and adds a user corresponding to the provider certificate in a node of the second instance;

[0015] The data providing device provides data to the data consuming device through the second instance based on the provider certificate and the contract agreement.

[0016] In one embodiment, the data providing device providing data to the data consuming device through the second instance based on the provider certificate and the contract agreement includes:

[0017] Determine the data compression format;

[0018] Based on the data compression format, compressing the data provided by the data consumption device to obtain compressed data;

[0019] Based on the provider certificate and the contract agreement, the compressed data is provided to the data consumption device through the second instance.

[0020] In one embodiment, after the data consuming device obtains the data provided by the data providing device through the first instance based on the contract agreement, the method further includes:

[0021] The data consumption device obtains the verification algorithm and verification code corresponding to the data provided by the data providing device;

[0022] The data consumption device calculates a hash value corresponding to the data provided by the data providing device based on the verification algorithm;

[0023] The data consumption device obtains the key header, key tail and private key fields sent by the data providing device;

[0024] The data consumption device assembles a verification code based on the key header, the key tail, the private key field and the hash value;

[0025] If the check code matches the verification code, it is determined that the data provided by the data providing device is normal.

[0026] In one embodiment, the data consumption device assembling the verification code based on the key header, the key tail, and the hash value includes:

[0027] Obtaining a middle field of the hash value and remaining fields except the middle field;

[0028] Combining the middle field with the key header to form a target key header;

[0029] Combining the middleware field and the key tail into a target key tail;

[0030] Inserting the remaining field into the private key field to obtain the inserted private key;

[0031] A verification code is assembled according to the target key header, the target key tail, and the inserted private key.

[0032] In one embodiment, inserting the remaining field into the private key field to obtain the inserted private key includes:

[0033] The remaining fields are sequentially inserted between two adjacent fields in the private key field to obtain the inserted private key.

[0034] In one embodiment, the data providing device providing data to the data consuming device through the second instance based on the contract agreement includes:

[0035] Based on the contract agreement, obtaining target data provided to the data consumption device from a data source;

[0036] Grouping the target data to obtain multiple groups of data;

[0037] Based on the number of groups, determining the number of concurrent threads;

[0038] Based on the number of concurrent threads, the multiple sets of data are concurrently provided to the data consumption device through the second instance.

[0039] In one embodiment, the data consuming device obtaining the data provided by the data providing device through the first instance based on the contract agreement includes:

[0040] The data providing device obtains the consumer certificate sent by the data consuming device; wherein the consumer certificate is used to authorize the data providing device to access;

[0041] The data providing device imports the consumer certificate into the information database and adds the user corresponding to the consumer certificate in the first instance;

[0042] The data consuming device obtains the data provided by the data providing device through the first instance based on the consumer certificate and the contract agreement.

[0043] In one embodiment, the data consuming device obtaining the data provided by the data providing device through the first instance based on the contract agreement includes:

[0044] An Output Port component is added to the input end of the data consumption device; wherein the Output Port component is used to receive data pushed by the output port of the data providing device.

[0045] According to another aspect of the present invention, there is provided a data exchange device, comprising:

[0046] A data providing device is configured to create a contract offer, wherein the contract offer represents a binding proposal from the data providing device; based on the proposed amendments, negotiate with the data consuming device and determine a contract agreement; construct a first instance; and based on the contract agreement, provide data to the data consuming device via the second instance;

[0047] A data consumption device is used to obtain the asset catalog sent by the data providing device and propose modification suggestions based on the contract offer; construct a second instance; wherein the first instance and the second instance are communicatively connected; based on the contract agreement, obtain the data provided by the data providing device through the first instance.

[0048] The data exchange method and device provided by the present invention include: a data providing device creates a contract offer, wherein the contract offer represents a binding proposal made by the data providing device; a data consuming device obtains an asset catalog sent by the data providing device and proposes amendments based on the contract offer; the data providing device negotiates with the data consuming device based on the amendments and determines a contract agreement; the data providing device constructs a first instance; the data consuming device constructs a second instance, wherein the first instance and the second instance are communicatively connected; the data consuming device obtains data provided by the data providing device through the first instance based on the contract agreement; and the data providing device provides data to the data consuming device through the second instance based on the contract agreement. By establishing a contract agreement between the data providing device and the data consuming device, and then setting instances at the data level, secure and reliable data transmission between the data providing device and the data consuming device is ensured based on the contract agreement. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other objects, features, and advantages of the present invention will become more apparent through a more detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. The accompanying drawings are provided to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and are not intended to limit the present invention. In the drawings, the same reference numerals generally represent the same components or steps.

[0050] Figure 1 It is a flowchart of a data exchange method provided by an exemplary embodiment of the present invention.

[0051] Figure 2 The figure is a flow chart of a data exchange method between a data providing device and a data consuming device provided by an exemplary embodiment of the present invention.

[0052] Figure 3It is a flowchart of a data exchange method provided by another exemplary embodiment of the present invention.

[0053] Figure 4 It is a structural diagram of a data exchange device provided by an exemplary embodiment of the present invention.

[0054] Figure 5 is a structural diagram of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0055] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0056] Figure 1 It is a flowchart of a data exchange method provided by an exemplary embodiment of the present invention. Figure 2 The figure is a flow chart of a data exchange method between a data providing device and a data consuming device provided by an exemplary embodiment of the present invention. Figure 3 FIG. 1 is a flow chart of a data exchange method provided by another exemplary embodiment of the present invention. Figure 1-Figure 3 As shown, the data exchange method includes:

[0057] Step 110: The data providing device creates a contract offer, wherein the contract offer represents a binding proposal made by the data providing device.

[0058] In this embodiment of the present invention, a control layer is constructed to facilitate cooperative negotiation between a data provider and a data consumer. The data provider creates a contract offer, which ensures that the two devices can negotiate a cooperative agreement. Based on the cooperative agreement, data exchange between the two devices is guaranteed.

[0059] Specifically, the data provider creates a contract offer for the data asset. This clearly defines the terms and conditions for data use, allowing the data provider and the data consumer to negotiate data usage, protection, and related responsibilities. This contract offer helps secure the rights and interests of both parties, build trust, and mitigate potential legal disputes.

[0060] Step 120: The data consumption device obtains the asset catalog sent by the data providing device and proposes modification suggestions based on the contract offer.

[0061] In an embodiment of the present invention, the data consumption device can modify the contract offer for the asset catalog sent by the data providing device, and can put forward its own requirements based on the contract offer, so that the data providing device can consider the requirements as appropriate, and the data providing device can also put forward its own opinions based on the requirements.

[0062] The data provider creates data assets and maintains an internal resource list, namely a data directory or asset directory. Then, an access policy is set for the asset directory, and the data consumer accesses the data assets corresponding to the data provider according to the access policy.

[0063] Furthermore, the data provider can obtain the access role of the data consumer; based on the access role, the data provider provides data to the data consumer. For example, if the access role is data analyst, then the user can read customer transaction records but cannot modify or delete any data.

[0064] Furthermore, the access policy also includes access time. The cooperation agreement can specify the access time of the data consumption device. For example, the cooperation agreement stipulates that the data consumption device can access between 11:00-18:00.

[0065] Step 130: The data providing device negotiates with the data consuming device based on the modification suggestions and determines a contract agreement.

[0066] In an embodiment of the present invention, the data provider negotiates based on the proposed changes and notifies the data consumer of the results. If the data consumer agrees, the data provider and the data consumer can then create a contract agreement based on the results. The contract agreement specifies the type and quantity of data to be exchanged between the data provider and the data consumer.

[0067] Step 140: The data providing device constructs a first instance.

[0068] In an embodiment of the present invention, Apache NiFi, as a powerful data stream processing tool, enables automated data transmission, conversion, and processing. NiFi allows users to fine-tune data flows, including operations such as data routing, conversion, filtering, distribution, and aggregation. It supports data encryption, access control, and authentication mechanisms to ensure data security during transmission and processing. It provides an end-to-end data confirmation mechanism to ensure reliable data transmission and processing. It can run on multiple nodes to process large-scale data streams. Therefore, in the data layer of the present invention, the data providing device constructs a first instance, which can be a NiFi instance.

[0069] Among them, the data provider configures the data source through the NiFi instance, supports structured, semi-structured, and unstructured types such as databases, files, HTTP interfaces, and supports both limited and unlimited data sets.

[0070] Step 150: The data consumption device constructs a second instance, wherein the first instance and the second instance are communicatively connected.

[0071] In an embodiment of the present invention, the second instance may be a NiFi instance. It is assumed that the first instance and the second instance can be communicatively connected, so that in data exchange, the data provider transmits data to the second instance through the first instance, and the second instance transmits the data to the data consumer for application by the data consumer.

[0072] Specifically, the data provider constructs a first connector, the data consumer constructs a second connector, and the data provider registers its identity in the second connector, while the data consumer registers its identity in the first connector. The data provider then registers its own NiFi instance with the data consumer, and the data consumer registers its own NiFi instance with the data provider. This ensures that the data provider and the data consumer can exchange data via the first and second instances of the communication connection.

[0073] Step 160: The data consuming device obtains the data provided by the data providing device through the first instance based on the contract agreement.

[0074] Step 170: The data providing device provides data to the data consuming device through the second instance based on the contract agreement.

[0075] In one embodiment, step 170 can be specifically implemented as follows: the data consumption device obtains the provider certificate sent by the data providing device; wherein the provider certificate is used to authorize the data providing device to access; the data consumption device imports the provider certificate into the trust store and adds the user corresponding to the provider certificate in the node of the second instance; the data providing device provides data to the data consumption device through the second instance based on the provider certificate and the contract agreement.

[0076] In an embodiment of the present invention, to ensure secure communication between a data consuming device and a data providing device, the data consuming device and the data providing device each obtain a server certificate and corresponding private key issued by a trusted CA. The server certificate issued by the trusted CA is referred to as the provider certificate. The provider certificate is associated with a private key, thereby ensuring that the data consuming device can grant access rights to the data providing device to obtain data provided by the data providing device.

[0077] Furthermore, the provider certificate, private key and CA root certificate are imported into the Java keystore or truststore of the data consumption device. When the data providing device needs to access the data consumption device, the provider private key sent is matched with the private key in the data consumption device. When the match is successful, the data providing device can access the data consumption device.

[0078] Furthermore, after the data consumption device obtains the provider certificate and imports it into the trust store, it adds a user named as the Common Name (CN) attribute of the provider certificate in the NiFi node of the second instance, such as CN=provider,OU=NIFI, and authorizes this certificate user to allow the provider to access, and grants it through the NiFi authorization module.<retrieve site-to-site details> Permission to allow provider users to retrieve the external service port provided by the consumer for subsequent pushing of data to the consumer.

[0079] In one embodiment, step 170 can be specifically implemented as follows: determining a data compression format; compressing the data provided by the data consumption device based on the data compression format to obtain compressed data; and providing the compressed data to the data consumption device through a second instance based on the provider certificate and the contract agreement.

[0080] In the embodiment of the present invention, the data providing device needs to connect to the data source before providing the data to the data consuming device. In other words, the data source information needs to be configured so that the data providing device can call the data in the data source.

[0081] Specifically, the QueryDatabaseTableRecord component is used to connect to the data source to obtain data. It supports querying specified tables, custom SQL queries, full queries, incremental queries, etc. Then, the data source connection properties are configured, which includes selecting the HikariCPConnectionPool component and filling in the required parameters of the database connection URL, database driver class name, database user name, and password. Optionally, the maximum number of connections to the data source (Max Total Connections) and the maximum wait time (Max WaitTime) are configured.

[0082] At the same time, configure the query table name (Table Name), column names (Columns to Return), and query conditions (Additional WHERE clause), or directly configure the query custom SQL (Custom Query). According to the contract agreement, if the data consumption device requires incremental data, that is, an unlimited data set, it is necessary to configure the incremental field (Maximum-valueColumns).

[0083] The data consumption device connects to the data source and imports data based on the PutDatabaseRecord component, supporting operations such as INSERT, UPDATE, and UPSERT. Specifically, configure the data source connection properties (Database Connection Pooling Service): select the HikariCPConnectionPool component, fill in the required parameters of the database connection address (Database Connection URL), database driver class (Database Driver Class Name), username (Database User), and password (Password), and optionally configure the maximum number of connections to the data source (MaxTotal Connections) and the maximum wait time (Max Wait Time); configure the imported table name (TableName); configure the operation type (Statement Type), which can be INSERT, UPDATE, or UPSERT; when the operation type is UPDATE or UPSERT, you need to configure the updated primary key (Update Keys).

[0084] The QueryDatabaseTableRecord component is a component used to query database table records in data processing and integration tools (such as Talend or similar ETL tools). It is typically used to extract data from a database.

[0085] The HikariCPConnectionPool component is a component used to manage database connection pools, commonly used in Java applications. HikariCP is a high-performance JDBC connection pool library, known for its speed and lightweight, and widely used in various Java applications and frameworks.

[0086] After determining the data required by the data consumer, the data provider compresses the data using the CompressContent component to improve the data transmission speed, reduce bandwidth requirements, and thus lower data transmission costs. The compression mode attribute is set to "compress." The compression algorithm attribute selects algorithms such as gzip, snappy, and zstd.

[0087] The CompressContent component is commonly used in data integration and processing tools (such as Talend or similar ETL tools). Its main function is to compress data to save storage space and improve transmission efficiency. This component can process data in various formats, such as text and binary files.

[0088] Similarly, when the data consumption device obtains the data provided by the data providing device, it needs to decompress the data based on the CompressContent component. Among them, the decompression mode is the "decompress" decompression mode, and the decompression algorithm can select gzip, snappy, zstd and other algorithms. The "Compress" compression mode refers to the process of compressing data to reduce the storage space it occupies or improve the efficiency of data transmission. The data compression format in the present invention includes lossless compression format and lossy compression format. According to the requirements of the contract agreement, a suitable compression method is selected to compress the data. Lossless compression formats include ZIP, TAR, etc., and lossy compression formats include JPEG, MP3, etc.

[0089] Gzip is a widely used compression algorithm based on the DEFLATE algorithm, primarily used for file compression and data transfer. Decompression mode restores compressed data to its original state. Snappy is a compression algorithm designed to provide very fast compression and decompression speeds at the expense of a certain compression ratio. Zstandard is a modern compression algorithm designed to provide a high compression ratio and fast compression / decompression speeds.

[0090] In one embodiment, after step 160, the data exchange method can be specifically implemented as follows: the data consumption device obtains the verification algorithm and verification code corresponding to the data provided by the data providing device; the data consumption device calculates the hash value corresponding to the data provided by the data providing device based on the verification algorithm; the data consumption device obtains the key header, key tail and private key fields sent by the data providing device; the data consumption device assembles a verification code based on the key header, key tail, private key fields and hash value; if the verification code matches the verification code, it is determined that there is no abnormality in the data provided by the data providing device.

[0091] In an embodiment of the present invention, to ensure that the received data has not been altered and originates from the provider of the message, the data providing device sets a check code and a verification algorithm when transmitting data. Through the check code and the verification algorithm, the data consuming device can determine whether the transmitted data has any abnormalities. Abnormal situations include, for example, data being altered. Among them, the data can be verified using the VerifyContentPGP component to ensure that the data has not been altered and originates from the provider of the message. The VerifyContentPGP component is generally related to the PGP (Pretty Good Privacy) encryption standard and is mainly used to verify the integrity and source of content.

[0092] Specifically, the data consuming device calculates a hash value corresponding to the data provided by the data providing device using a verification algorithm. Verification algorithms may include SHA256, SHA384, SHA512, and other algorithms. The device then obtains the key header, key tail, and private key fields sent by the data providing device and assembles a verification code using these fields and the hash value. The verification code sent by the data providing device is then compared. If the verification code matches the verification code, the data provided by the data providing device is deemed to be normal.

[0093] The data provider can use the StandardPGPPrivateKeyService component to fill in the private key storage file attribute (Keyring File) or the private key content attribute (Keyring) and fill in the private key. The StandardPGPPrivateKeyService component is usually related to PGP (Pretty Good Privacy) encryption and decryption, and is used to handle the management and operation of private keys.

[0094] In addition, when the data consumption device in the present invention verifies whether the data provided by the data providing device is safe, has not been changed and originates from the message provider, it can configure the public key service attributes (Public Key Service). Specifically, it selects the StandardPGPPublicKeyService component and fills in the public key storage file attributes (Keyring File) or the public key content attributes (Keyring).

[0095] In one embodiment, the data exchange method can be specifically implemented as follows: obtaining the middle field of the hash value and the remaining fields except the middle field; combining the middle field with the key header to form a target key header; combining the middle field with the key tail to form a target key tail; inserting the remaining fields into the private key field to obtain the inserted private key; assembling the verification code based on the target key header, the target key tail and the inserted private key.

[0096] In this embodiment of the present invention, since the data providing device may be easily tampered with when transmitting data to the data consuming device, the present invention effectively prevents data tampering by processing different parts of the hash value (the middle field and the remaining field) separately. If any part is modified, the verification code will no longer be valid, thereby detecting data integrity issues.

[0097] Specifically, suppose we have a hash value of abcdef1234567890. We can define the middle field as 1234 and the remaining fields as abcdef7890. Setting the header to "HEADER_" will result in the target header being "HEADER_1234." Setting the tail to "_FOOTER" will result in the target tail being "1234_FOOTER." Setting the private key field to "PRIVATE_KEY" will result in the inserted private key being "PRIVATE_KEYabcdef7890." The final verification code will be "HEADER_1234PRIVATE_KEYabcdef7890 1234_FOOTER."

[0098] In one embodiment, the data exchange method may be specifically implemented as follows: inserting the remaining fields in sequence between two adjacent fields in the private key field to obtain the inserted private key.

[0099] In an embodiment of the present invention, two adjacent fields in the private key field are determined, and the private key field can be divided based on the meaning of English words. For example, if the private key field is "PRIVATE_KEY", then it can be considered to be composed of two parts, "PRIVATE" and "KEY". The remaining field is "abcdef7890", which needs to be inserted between PRIVATE and KEY. Insert the characters of the remaining field one by one between the two parts of the private key field. Insert the first character 'a' of the remaining field between PRIVATE and KEY, and the result is: PRIVATEaKEY. Insert the second character 'b' of the remaining field between PRIVATEa and KEY, and the result is: PRIVATEabKEY, and so on, and the final inserted key is "PRIVATEabcdef7890KEY".

[0100] In one embodiment, step 170 can be specifically implemented as follows: based on the contract agreement, obtaining the target data provided to the data consumption device from the data source; grouping the target data to obtain multiple groups of data; based on the number of groups, determining the number of concurrent threads; based on the number of concurrent threads, concurrently providing multiple groups of data to the data consumption device through the second instance.

[0101] In an embodiment of the present invention, target data provided by a data consumption device is obtained from a data source through a contract agreement, and then the target data is grouped, and the number of concurrent threads is determined based on the number of groups. Based on the number of concurrent threads, the data providing device provides concurrent threads that meet the number of concurrent threads and uses concurrent threads to transmit data to the data consumption device.

[0102] The present invention is based on the Input Port component. The consumer side adds a Remote connections (site-to-site) type Input Port component at the entrance of the above data output process to receive data pushed by the provider. The number of concurrent threads (Concurrent Tasks) receiving data can be configured to improve the data transmission speed. At the same time, the component needs to be configured with a policy to grant the provider user<receive data via site-to-site> Permissions ensure that only the provider side specified in the contract can retrieve the component for pushing data.

[0103] In addition, the data providing device in the present invention is based on the Remote Process Group component. The provider side adds the Remote Process Group component at the exit of the above data input process, configures the address of the consumer side NiFi node, and selects the data receiving port of the above consumer side to transmit data.

[0104] Among them, Input Port is an interface in the system for receiving external data streams. Remote connections (site-to-site) allow direct data transmission between two different locations or systems. Receive data via site-to-site refers to receiving data through the above-mentioned remote connection method. This means that data is sent from one site (for example, a data center or cloud service) to the Input Port component of another site through a secure connection. The Remote Process Group (RPG) component is an important component in data flow management tools such as Apache NiFi. It is used to manage and coordinate data flows in a distributed environment, allowing communication and data transmission between different NiFi instances.

[0105] Similarly, the data consumption device is based on the Output Port component. The provider adds a Remote connections (site-to-site) type Output Port component at the exit of the above data input process to receive data pushed by the provider. The number of concurrent threads (Concurrent Tasks) receiving data can be configured to increase the data transmission speed. At the same time, the component needs to be configured with a policy to grant the consumer user<receive data via site-to-site> Permissions ensure that only the consumer side specified in the contract can retrieve the component to receive data.

[0106] Furthermore, the data consumption device is based on the Remote Process Group component. The consumer adds the Remote Process Group component to the data output process entry, configures the address of the provider's NiFi node, and selects the provider's data output port to receive data. The Output Port is a special component in NiFi, primarily used to transfer data from one data stream to another component or system. The Remote Process Group (RPG) is a key component in Apache NiFi, used to connect and transfer data streams between different NiFi instances.

[0107] In one embodiment, step 160 can be specifically implemented as follows: the data providing device obtains the consumer certificate sent by the data consuming device; wherein the consumer certificate is used to authorize the data providing device to access; the data providing device imports the consumer certificate into the information library and adds the user corresponding to the consumer certificate in the first instance; the data consuming device obtains the data provided by the data providing device through the first instance based on the consumer certificate and the contract agreement.

[0108] In an embodiment of the present invention, to ensure secure communication between a data consuming device and a data providing device, the data consuming device and the data providing device each obtain a server certificate and corresponding private key issued by a trusted CA. The server certificate issued by the trusted CA is referred to as the provider certificate. The provider certificate is associated with a private key, thereby ensuring that the data consuming device can grant access rights to the data providing device to obtain data provided by the data providing device.

[0109] In one embodiment, step 160 may be specifically implemented as: adding an OutputPort component to the input end of the data consumption device; wherein the OutputPort component is used to receive data pushed by the output port of the data providing device.

[0110] Figure 4FIG. 1 is a structural diagram of a data exchange device provided by an exemplary embodiment of the present invention. Figure 4 As shown, the data exchange device includes: a data providing device, which is used to create a contract offer; wherein the contract offer represents a binding proposal made by the data providing device; based on the modification opinions, negotiate with the data consuming device and determine the contract agreement; construct a first instance; based on the contract agreement, provide data to the data consuming device through the second instance; the data consuming device, which is used to obtain the asset catalog sent by the data providing device and propose modification opinions based on the contract offer; construct a second instance; wherein the first instance and the second instance are communicatively connected; based on the contract agreement, obtain the data provided by the data providing device through the first instance.

[0111] Figure 5 The figure shows a block diagram of an electronic device according to an embodiment of the present application.

[0112] like Figure 5 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0113] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0114] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the data exchange method of each embodiment of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0115] In one example, the electronic device 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0116] When the electronic device 10 is a stand-alone device, the input device 13 may be a communication network connector, configured to receive collected input signals from the first device and the second device.

[0117] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0118] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0119] Of course, to simplify, Figure 5 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 10 may further include any other appropriate components according to specific application scenarios.

[0120] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0121] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data exchange method, characterized in that: include: The data providing device creates a contract offer; wherein the contract offer represents a binding proposal made by the data providing device; The data consumption device obtains the asset catalog sent by the data providing device and proposes modification suggestions based on the contract offer; The data providing device negotiates with the data consuming device based on the modification suggestions and determines a contract agreement; The data providing device constructs a first instance; The data consumption device establishes a second instance; wherein the first instance and the second instance are communicatively connected; The data consumption device obtains the data provided by the data providing device through the first instance based on the contract agreement; The data providing device provides data to the data consuming device through the second instance based on the contract agreement.

2. The data exchange method according to claim 1, characterized in that: The data providing device providing data to the data consuming device through the second instance based on the contract agreement includes: The data consumption device obtains the provider certificate sent by the data providing device; wherein the provider certificate is used to authorize the data providing device to access; The data consumption device imports the provider certificate into a trust store and adds a user corresponding to the provider certificate in a node of the second instance; The data providing device provides data to the data consuming device through the second instance based on the provider certificate and the contract agreement.

3. The data exchange method according to claim 2, characterized in that: The data providing device providing data to the data consuming device through the second instance based on the provider certificate and the contract agreement includes: Determine the data compression format; Based on the data compression format, compressing the data provided by the data consumption device to obtain compressed data; Based on the provider certificate and the contract agreement, the compressed data is provided to the data consumption device through the second instance.

4. The data exchange method according to claim 1, wherein: After the data consumption device obtains the data provided by the data providing device through the first instance based on the contract agreement, the data consumption device further includes: The data consumption device obtains the verification algorithm and verification code corresponding to the data provided by the data providing device; The data consumption device calculates a hash value corresponding to the data provided by the data providing device based on the verification algorithm; The data consumption device obtains the key header, key tail and private key fields sent by the data providing device; The data consumption device assembles a verification code based on the key header, the key tail, the private key field and the hash value; If the check code matches the verification code, it is determined that the data provided by the data providing device is normal.

5. The data exchange method according to claim 4, characterized in that: The data consumption device assembling the verification code based on the key header, the key tail, and the hash value includes: Obtaining a middle field of the hash value and remaining fields except the middle field; Combining the middle field with the key header to form a target key header; Combining the middleware field and the key tail into a target key tail; Inserting the remaining field into the private key field to obtain the inserted private key; A verification code is assembled according to the target key header, the target key tail, and the inserted private key.

6. The data exchange method according to claim 1, characterized in that: Inserting the remaining fields into the private key field to obtain the inserted private key includes: The remaining fields are sequentially inserted between two adjacent fields in the private key field to obtain the inserted private key.

7. The data exchange method according to claim 1, characterized in that: The data providing device providing data to the data consuming device through the second instance based on the contract agreement includes: Based on the contract agreement, obtaining target data provided to the data consumption device from a data source; Grouping the target data to obtain multiple groups of data; Based on the number of groups, determining the number of concurrent threads; Based on the number of concurrent threads, the multiple sets of data are concurrently provided to the data consumption device through the second instance.

8. The data exchange method according to claim 1, characterized in that: The data consuming device acquiring the data provided by the data providing device through the first instance based on the contract agreement includes: The data providing device obtains the consumer certificate sent by the data consuming device; wherein the consumer certificate is used to authorize the data providing device to access; The data providing device imports the consumer certificate into the information database and adds the user corresponding to the consumer certificate in the first instance; The data consuming device obtains the data provided by the data providing device through the first instance based on the consumer certificate and the contract agreement.

9. The data exchange method according to claim 1, characterized in that: The data consuming device acquiring the data provided by the data providing device through the first instance based on the contract agreement includes: An Output Port component is added to the input end of the data consumption device; wherein the Output Port component is used to receive data pushed by the output port of the data providing device.

10. A data exchange device, characterized in that: include: A data providing device is configured to create a contract offer, wherein the contract offer represents a binding proposal from the data providing device; based on the proposed amendments, negotiate with the data consuming device and determine a contract agreement; construct a first instance; and based on the contract agreement, provide data to the data consuming device via the second instance; A data consumption device is used to obtain the asset catalog sent by the data providing device and propose modification suggestions based on the contract offer; construct a second instance; wherein the first instance and the second instance are communicatively connected; based on the contract agreement, obtain the data provided by the data providing device through the first instance.