A data interconnection and intercommunication implementation method based on a data connector

Abstract

The application relates to the technical field of data interconnection and intercommunication, and discloses a data interconnection and intercommunication implementation method based on a data connector, which comprises the following steps: an access connector containing an information exchange model and a protocol adapter is assembled, heterogeneous protocols such as HTTP / HTTPS and AMQP are adapted and converted into unified standard protocols; data directory registration and release of the access connector are completed, and are synchronized to business-area-global nodes to form a complete data directory and an API privacy authorization interface resource pool; a third party initiates a request after authentication, and realizes standardized data transmission by relying on four core capabilities and three layers of nodes; and in the fourth step, a message retransmission mechanism combining exponential backoff and random jitter and a whole-process log are used to guarantee interaction reliability and traceability. The application saves repeated development of heterogeneous system docking, reduces cost and improves efficiency, supports system expansion, improves data asset transparency and service discovery rate, reduces resource occupation, shortens transmission delay, and efficiently supports large-scale cross-subject data interconnection and intercommunication.

Description

Technical Field

[0001] This invention relates to the field of data interconnection technology, and more specifically, to a data interconnection implementation method based on a data connector. Background Technology

[0002] In today's rapidly developing digital economy, data has become a core production factor, and cross-system and cross-domain data interconnection has become a key requirement for enterprise digital transformation and industry collaborative development. Whether in government services, financial transactions, or industrial manufacturing, various business systems need to achieve business collaboration through data sharing, but current data interconnection still faces multiple technical bottlenecks.

[0003] In existing technologies, the interface protocols of various business systems are not standardized. When heterogeneous systems using the AMQP protocol need to exchange data with systems using the SOAP protocol, a large amount of additional adaptation code needs to be developed, resulting in high costs and long cycles for system integration. At the same time, the lack of a standardized protocol conversion mechanism makes data prone to formatting errors and loss during transmission, seriously affecting the efficiency of data exchange.

[0004] Existing data directories are mostly scattered across various business systems, lacking a centralized directory management and synchronization mechanism. Users struggle to quickly locate the data resources they need, and directory information is often not updated in a timely manner, easily leading to data silos. Furthermore, data directories are not deeply integrated with access control, failing to implement fine-grained access control based on data sensitivity, thus posing a risk of data leakage. Therefore, there is an urgent need to provide a data interconnection and interoperability implementation method based on data connectors to solve these problems. Summary of the Invention

[0005] To address the aforementioned technical challenges, this invention provides a data interconnection and interoperability implementation method based on a data connector. By adapting heterogeneous protocols through the access connector, it eliminates redundant development, reduces costs and improves efficiency while supporting system expansion; multi-mode synchronization of the data catalog enhances asset transparency and service discovery rate; national cryptographic algorithms and fine-grained control ensure security and compliance; a message retransmission mechanism reduces resource consumption and improves transmission reliability; and three-layer node routing shortens latency, efficiently supporting large-scale cross-entity data interconnection and interoperability.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A method for achieving data interconnection and interoperability based on data connectors includes the following steps:

[0008] S1. The business system provides an external access interface for at least one of the following protocols: HTTP / HTTPS, AMQP, FTP / SFTP, and SOAP, and selects the optimal protocol method; an access connector is constructed consisting of an information exchange model and a protocol adapter. The information exchange model standardizes and encapsulates the business system's protocol interfaces and abstracts and describes business data assets. The protocol adapter converts heterogeneous protocols into a unified standard protocol.

[0009] S2. The business system reports the data catalog information to the regional functional node and binds the access connector to initiate registration. After the data catalog is published by the business functional node, it is synchronized to the regional and global functional nodes to form a complete data catalog. The access connector completes the registration through the regional functional node and generates an authorization license identification code. The registration parameters are initialized when the message flow engine starts and remain in memory. The access connector encapsulates the business data service and forms a standard interface after privacy computation audit. It allocates access and subscription permissions and publishes them to the outside world, forming an API privacy authorization interface data catalog resource pool.

[0010] S3: After authentication, third-party business systems browse the data directory and initiate data requests. The request and response data are encapsulated into XML or JSON format message messages through the information exchange model, flow through the message queue, and are distributed to the target functional unit after being processed by the message stream. Relying on the four core capabilities of data publishing and discovery, data interconnection, spatial digital identity, and data exchange model, directory query, protocol and semantic and structural conversion, full-process security protection, and standardized data transmission are completed. Routing and data transmission are realized through three-layer functional nodes of business, region, and global domain.

[0011] S4. The message retransmission mechanism is executed through the message queue manager. When a message fails to be sent or times out and the maximum number of retransmissions has not been reached, it is retransmitted using an exponential backoff combined with a random jitter strategy. If the message still fails after reaching the maximum number of retransmissions, an alarm is triggered. The message queue manager monitors queue instances and dynamically adjusts queue resources. Data interaction traceability and abnormal operation location are achieved by using full-process logs and security protection measures.

[0012] As a preferred embodiment of the present invention, the FTP / SFTP protocol is selected for large data transmission scenarios in S1, and the AMQP protocol is selected for real-time interaction scenarios; the abstract description of the business data assets includes information source, information format, exchange protocol and path selection address.

[0013] As a preferred embodiment of the present invention, the complete data catalog in S2 includes data name, data type, business system to which it belongs, update time, sensitivity level, information format, protocol and content example; the privacy computation adopts a federated learning algorithm.

[0014] As a preferred embodiment of the present invention, the message message in S3 includes a service identifier field, a service data field, and a response information field, wherein the service identifier field includes SourceSysID and ServiceID, the response information field includes Status and Code, and the message structure conforms to a preset message standard.

[0015] As a preferred embodiment of the present invention, the data publishing and discovery capability described in S3 supports multi-condition combination queries and fuzzy searches based on keywords, categories, and time ranges. Directory synchronization adopts a timed, real-time, or incremental mode and has synchronization status monitoring and abnormal alarm functions. For sensitive data directories, privacy computing encryption algorithms such as federated learning encryption and homomorphic encryption are used to desensitize and anonymize privacy information, set fine-grained access control rules, and record permission operation logs.

[0016] As a preferred embodiment of the present invention, the data interconnection capability described in S3 configures access permissions based on the access connector registration information and data sensitivity, verifies the identity and permissions of the requester in real time and blocks unauthorized or cross-permission access; automatically identifies heterogeneous protocols such as HTTP / HTTPS, AMQP, SOAP, FTP / SFTP and completes the conversion, realizes semantic conversion through preset mapping rules or custom mapping tables, and reorganizes the source data structure according to the requirements of the target system; and automatically collects key information logs throughout the data interaction process and stores them in the regional functional nodes.

[0017] As a preferred embodiment of the present invention, the spatial digital identity capability described in S3 employs multiple encryption algorithms such as SM4 symmetric encryption and SM2 asymmetric encryption, combined with OAuth2.0, SAML and other identity authentication protocols and dynamic key management mechanisms to ensure identity security; data transmission uses SM4 encryption, storage uses SM3 hash encryption, privacy information is desensitized and field-level fine-grained permission control is implemented, and keys are updated regularly; security assessments and access controls such as vulnerability scanning and code auditing are performed on application access, data call and operation behavior are monitored in real time, and warnings are triggered and blocked when anomalies occur; SSL / TLS transmission encryption protocol and IPsec secure tunnel technology based on national cryptographic algorithms are deployed, the transmission link status is monitored 24 hours a day, and a defense mechanism is activated when anomalies occur.

[0018] As a preferred embodiment of the present invention, the data exchange model in S3 consists of a catalog list and a component manager, and its structure is divided into an identifier field and a data field; the catalog list records asset metadata such as data name and data type; the component manager gives business data service-oriented information communication capabilities; the identifier field includes information such as model version number and transmission priority; the data field is divided into stream data that records request stream information and business data that records exchanged data content.

[0019] As a preferred embodiment of the present invention, the routing and transmission process described in S3 is as follows: after the message is parsed through the message flow pipeline, it is matched with the registration data to obtain routing information; the business function node connects to the business system and is responsible for data transmission, log collection and basic security verification. When an illegal data connector is detected, the transmission is blocked and the information is reported to the regional function node; the regional function node summarizes and analyzes the information and reports it to the global function node, while managing the registration of data connectors, directory synchronization and permission allocation within its region; the global function node manages all regional function nodes in a coordinated manner, controls and schedules data connector transaction actions, and maintains the global data directory and security policy.

[0020] As a preferred embodiment of the present invention, the message retransmission mechanism in S4 includes variable definition, triggering condition function, retransmission interval formula, and execution logic function, as detailed below:

[0021] (1) Variable definition:

[0022] The service message to be transmitted is a business data message in XML or JSON format;

[0023] Message queue manager, responsible for message scheduling and retransmission control;

[0024] :information The sending status, This indicates that the message was sent successfully. This indicates that the transmission failed. Indicates a timeout;

[0025] Message sending timeout threshold, determined by Configure according to business scenarios;

[0026] :information Time elapsed from sending to receiving the response;

[0027] :information The number of retransmissions, initial value , ;

[0028] Maximum number of retransmissions, determined by... Dynamic configuration to avoid resource consumption;

[0029] : No. Second response and the first The time interval between each send / retransmission;

[0030] Initial retransmission interval, determined by Adjust dynamically based on network load;

[0031] Random jitter factor This is used to avoid the thunderous herd effect of multiple message retransmissions;

[0032] :information Alarm trigger status, This indicates that an alarm has been triggered. This indicates that no alarm will be triggered;

[0033] (2) Triggering condition function:

[0034] ;

[0035] in, Indicates triggering the first Second resend This indicates that resending has been terminated;

[0036] (3) Retransmission interval formula:

[0037]

[0038] in, This reflects an index retreat. It exhibits random jitter;

[0039] (4) Execute the logic function:

[0040] .

[0041] The beneficial technical effects of this invention are:

[0042] The solution features an access connector that includes an information exchange model and protocol adapter, automatically adapting to heterogeneous protocols such as HTTP / HTTPS, AMQP, SOAP, and FTP / SFTP. This eliminates the need for customized development for different systems, saving repetitive development work during heterogeneous system integration, significantly shortening the integration cycle and reducing development costs. Simultaneously, data transmission is free from formatting errors or loss, achieving extremely high conversion efficiency and preventing business interruptions due to data errors, thus greatly improving data interaction efficiency. Furthermore, this solution supports rapid integration of new protocols with a significantly shortened adaptation cycle, meeting the dynamic expansion needs of business systems.

[0043] Based on the data catalog registration and synchronization mechanism during the registration and release phases, the data catalog can be synchronized between business, regional, and global nodes in a scheduled, real-time, and incremental manner, significantly improving the transparency of data asset information; the success rate of cross-entity data service discovery is greatly improved, avoiding data interaction failures caused by lag in catalog information.

[0044] By leveraging the spatial digital identity capability design in the data interaction phase, and employing national cryptographic algorithms such as SM4 and SM2 to achieve end-to-end encryption of identity authentication, data transmission, and storage, combined with field-level desensitization and fine-grained access control, it fully complies with relevant regulatory requirements; the risk of data leakage is significantly reduced, and unauthorized access can be completely intercepted.

[0045] Through the message retransmission mechanism in the anomaly handling and protection phases, it adopts an exponential backoff combined with a random jitter strategy to effectively avoid the thunderous group effect of multiple message retransmissions, significantly reducing network resource consumption; the message transmission success rate is greatly improved, and the full-process log recording supports rapid anomaly location, significantly improving the efficiency of problem investigation.

[0046] Based on the three-layer node routing and transmission scheme in the data interaction stage, business-region-global node collaboration enables data rule-based routing, significantly reducing cross-regional data transmission latency; the collaboration efficiency between business systems is greatly improved, and it can support large-scale cross-entity data interconnection scenarios such as cross-departmental government affairs and cross-factory industrial areas. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the usage framework of the data connector of the present invention.

[0048] Figure 2 This is a schematic diagram illustrating the data connector's ability to enable business data transmission connectivity in this invention. Detailed Implementation

[0049] In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0050] Combination Figure 1-2 The present invention provides the following embodiments:

[0051] A method for achieving data interconnection and interoperability based on data connectors includes the following steps:

[0052] S1. The business system provides an external access interface for at least one of the following protocols: HTTP / HTTPS, AMQP, FTP / SFTP, and SOAP, and selects the optimal protocol method; an access connector is constructed consisting of an information exchange model and a protocol adapter. The information exchange model standardizes and encapsulates the business system's protocol interfaces and abstracts and describes business data assets. The protocol adapter converts heterogeneous protocols into a unified standard protocol.

[0053] S2. The business system reports the data catalog information to the regional functional node and binds the access connector to initiate registration. After the data catalog is published by the business functional node, it is synchronized to the regional and global functional nodes to form a complete data catalog. The access connector completes the registration through the regional functional node and generates an authorization license identification code. The registration parameters are initialized when the message flow engine starts and remain in memory. The access connector encapsulates the business data service and forms a standard interface after privacy computation audit. It allocates access and subscription permissions and publishes them to the outside world, forming an API privacy authorization interface data catalog resource pool.

[0054] S3: After authentication, third-party business systems browse the data directory and initiate data requests. The request and response data are encapsulated into XML or JSON format message messages through the information exchange model, flow through the message queue, and are distributed to the target functional unit after being processed by the message stream. Relying on the four core capabilities of data publishing and discovery, data interconnection, spatial digital identity, and data exchange model, directory query, protocol and semantic and structural conversion, full-process security protection, and standardized data transmission are completed. Routing and data transmission are realized through three-layer functional nodes of business, region, and global domain.

[0055] S4. The message retransmission mechanism is executed through the message queue manager. When a message fails to be sent or times out and the maximum number of retransmissions has not been reached, it is retransmitted using an exponential backoff combined with a random jitter strategy. If the message still fails after reaching the maximum number of retransmissions, an alarm is triggered. The message queue manager monitors queue instances and dynamically adjusts queue resources. Data interaction traceability and abnormal operation location are achieved by using full-process logs and security protection measures.

[0056] Furthermore, in S1, the FTP / SFTP protocol is selected for large data transmission scenarios, and the AMQP protocol is selected for real-time interaction scenarios; the abstract description of the business data assets includes information source, information format, exchange protocol, and path selection address.

[0057] FTP / SFTP protocols feature breakpoint resumption, batch transmission, and high bandwidth utilization, making them suitable for large data transmission scenarios with high data volume and long transmission cycles, reducing duplicate transmissions caused by interruptions. AMQP protocols support message acknowledgment, persistence mechanisms, and low-latency transmission, meeting the timeliness and reliability requirements of data transmission in real-time interactive scenarios. Abstracting and describing the core attributes of business data assets can form a unified metadata standard, providing precise attribute basis for protocol conversion and data adaptation of access connectors, ensuring accurate identification and parsing of data in subsequent flows.

[0058] Furthermore, the complete data catalog mentioned in S2 includes data name, data type, business system to which it belongs, update time, sensitivity level, information format, protocol, and content example; the privacy computation adopts a federated learning algorithm.

[0059] A complete data catalog encompasses the core attributes and usage-related information of the data. Basic attributes such as data name and type facilitate quick identification of data resources by users, while sensitivity levels provide a basis for subsequent access control. Content examples lower the barrier to data use. The data catalog, constructed through multi-dimensional attributes, enables refined management and efficient retrieval of data resources. Federated learning algorithms, by training models locally at each participating party and interacting only with model parameters, avoid the cross-system transmission of raw business data. This ensures compliance auditing of business data services while guaranteeing data privacy and security, and guarantees the security and compliance of interface releases.

[0060] Furthermore, the message message described in S3 includes a service identifier field, a service data field, and a response information field. The service identifier field contains SourceSysID and ServiceID, and the response information field contains Status and Code. The message structure conforms to the preset message standard.

[0061] The SourceSysID in the service identifier field is used to locate the source system of the data request, while the ServiceID precisely points to the target service interface. Together, they enable accurate routing of service calls. The service data field centrally carries core interactive data, ensuring the integrity of data transmission. The response information field provides intuitive feedback on the interaction status through parameters such as Status and Code, facilitating rapid identification of transmission anomalies. The standardized message structure eliminates differences in data formats between different systems, reduces parsing difficulty, and improves the compatibility and efficiency of cross-system data interaction.

[0062] Furthermore, the data publishing and discovery capabilities described in S3 support multi-condition combined queries and fuzzy searches based on keywords, categories, and time ranges. Directory synchronization adopts timed, real-time, or incremental modes and has synchronization status monitoring and anomaly alarm functions. Sensitive data directories employ privacy-preserving computation encryption algorithms such as federated learning encryption and homomorphic encryption to desensitize and anonymize privacy information, set fine-grained access control rules, and record permission operation logs.

[0063] Multi-condition combined queries and fuzzy searches expand the search scope and improve search accuracy through indexing mechanisms, meeting diverse user query needs. Scheduled synchronization adapts to scenarios with low data update frequencies, real-time synchronization ensures the consistency of highly time-sensitive data, and incremental synchronization transmits only updated data, reducing network bandwidth consumption. Privacy-preserving computing encryption algorithms encrypt sensitive data directories, using anonymization and desensitization techniques to hide the true content of private information. Fine-grained access control rules allocate data access scope according to user permissions, and combined with the traceability capabilities of permission operation logs, a comprehensive privacy protection system is formed throughout the entire lifecycle of sensitive data.

[0064] Furthermore, the data interconnection capability described in S3 configures access permissions based on the access connector registration information and data sensitivity, verifies the identity and permissions of the requester in real time, and blocks unauthorized or cross-permission access; automatically identifies heterogeneous protocols such as HTTP / HTTPS, AMQP, SOAP, and FTP / SFTP and completes the conversion, realizes semantic conversion through preset mapping rules or custom mapping tables, and reorganizes the source data structure according to the requirements of the target system; and automatically collects key information logs throughout the data interaction process and stores them in the regional functional nodes.

[0065] Based on access connector registration information and data sensitivity-based permission configuration, a correspondence between identity, permission, and data is established. The real-time verification mechanism can quickly identify unauthorized or cross-permission access behaviors, blocking security risks at the source. The protocol adapter automatically adapts to various heterogeneous protocols through built-in protocol parsing and conversion logic, eliminating protocol barriers. Preset mapping rules or custom mapping tables establish a correspondence between data semantics between different systems, achieving semantic uniformity. Data structure reorganization adapts the source data to the target system's receiving format, ensuring data availability. Full-process logging provides complete evidence for auditing data interaction and tracing anomalies, facilitating problem localization and responsibility determination.

[0066] Furthermore, the spatial digital identity capability described in S3 employs multiple encryption algorithms, such as SM4 symmetric encryption and SM2 asymmetric encryption, in accordance with national cryptographic standards. These are combined with identity authentication protocols such as OAuth2.0 and SAML, along with a dynamic key management mechanism, to ensure identity security. Data transmission uses SM4 encryption, and storage uses SM3 hash encryption. Privacy information is anonymized, and fine-grained field-level access control is implemented. Keys are updated regularly. Security assessments and access controls, such as vulnerability scanning and code auditing, are performed on application access. Data call and operation behaviors are monitored in real time, triggering warnings and blocking when anomalies occur. An SSL / TLS transmission encryption protocol and IPsec secure tunnel technology, optimized based on national cryptographic algorithms, are deployed to monitor the transmission link status 24 hours a day and activate defense mechanisms when anomalies occur.

[0067] SM4 symmetric encryption and SM2 asymmetric encryption algorithms possess the characteristics of independent control and high encryption strength. Combined with identity authentication protocols such as OAuth2.0 and SAML, they can achieve strong identity verification. The dynamic key management mechanism reduces the risk of key leakage by regularly updating keys. SM4 transmission encryption ensures that data is not eavesdropped on or tampered with during transmission, while SM3 hash encryption ensures the integrity and immutability of data storage. Field-level fine-grained permission control achieves precise protection of privacy information. Security assessment methods such as vulnerability scanning and code auditing can identify security risks in application access in advance, and real-time monitoring mechanisms can promptly detect abnormal data operation behavior. SSL / TLS and IPsec technologies build a secure transmission link, and multi-layered security protection measures form a comprehensive security protection system.

[0068] Furthermore, the data exchange model described in S3 consists of a catalog list and a component manager, with a structure divided into an identifier field (Header) and a data field (Body). The catalog list records asset metadata such as data name and data type. The component manager enables service-oriented information communication capabilities for business data. The identifier field contains information such as model version number and transmission priority. The data field is divided into stream data that records request flow information and business data that records exchanged data content.

[0069] The catalog list aggregates data asset metadata to form a unified index of data resources, providing a foundation for data querying and location. The component manager encapsulates service-oriented communication logic, enabling standardized interaction capabilities for business data and adapting to cross-system service call scenarios. The model version number in the identifier domain ensures compatibility between different model versions, and transmission priority enables differentiated scheduling of data transmission, ensuring that high-priority data is transmitted first. The data domain separates pipeline data from business data. Pipeline data records the transmission trajectory for easy traceability, while business data focuses on core interaction content to improve transmission efficiency. The combination of the two ensures both traceability and optimized transmission performance.

[0070] Furthermore, the routing and transmission process described in S3 is as follows: after the message is parsed through the message flow pipeline, it is matched with the registration data to calculate and obtain routing information; the business function node connects to the business system and is responsible for data transmission, log collection and basic security verification. When an illegal data connector is detected, the transmission is blocked and the information is reported to the regional function node; the regional function node summarizes and analyzes the information and reports it to the global function node, while managing the registration of data connectors, directory synchronization and permission allocation within its region; the global function node manages all regional function nodes in a coordinated manner, controls and schedules data connector transaction actions, and maintains the global data directory and security policies.

[0071] Message parsing and registration data matching calculations accurately determine data transmission paths by comparing message identifier information with registration metadata, avoiding routing confusion. Business function nodes, acting as an intermediary layer between business systems and core nodes, reduce direct coupling between them. Basic security checks can quickly filter out illegal connections, reducing the security burden on core nodes. Regional function nodes, through resource coordination within their regions, achieve localized data management and scheduling, improving data transmission efficiency within the region. They also aggregate information to global nodes, providing a basis for global scheduling. Global function nodes, as top-level control nodes, maintain global data directories and security policies, enabling unified scheduling and security control of cross-regional data transmission, forming a layered and orderly routing and transmission architecture.

[0072] Furthermore, the message retransmission mechanism described in S4 includes variable definitions, triggering condition functions, retransmission interval formulas, and execution logic functions, as detailed below:

[0073] (1) Variable definition:

[0074] The service message to be transmitted is a business data message in XML or JSON format;

[0075] Message queue manager, responsible for message scheduling and retransmission control;

[0076] :information The sending status, This indicates that the message was sent successfully. This indicates that the transmission failed. Indicates a timeout;

[0077] Message sending timeout threshold, determined by Configure according to business scenarios;

[0078] :information Time elapsed from sending to receiving the response;

[0079] :information The number of retransmissions, initial value This corresponds to the first transmission. ;

[0080] Maximum number of retransmissions, determined by... Dynamic configuration to avoid resource consumption;

[0081] : No. Second response and the first The time interval between each send / retransmission;

[0082] Initial retransmission interval, determined by Adjust dynamically based on network load;

[0083] Random jitter factor This is used to avoid the thunderous herd effect of multiple message retransmissions;

[0084] :information Alarm trigger status, This indicates that an alarm has been triggered. This indicates that no alarm will be triggered;

[0085] (2) Triggering condition function:

[0086] ;

[0087] in, Indicates triggering the first Second resend This indicates that resending has been terminated;

[0088] (3) Retransmission interval formula:

[0089]

[0090] in, This reflects the exponential retreat mechanism, where the more re-attempts are made, the greater the intervals between them, in order to reduce resource waste. It reflects random jitter and avoids overlapping resend times for multiple messages;

[0091] (4) Execute the logic function:

[0092] .

[0093] The variable definitions clearly define the function and value range of each parameter in the message retransmission mechanism, providing a clear data foundation for the mechanism's execution. The trigger condition function accurately filters scenarios requiring retransmission by judging the message sending status, time consumption, and number of retransmissions, avoiding invalid retransmissions that consume network resources. The exponential backoff strategy reduces the impact of high-frequency retransmissions on the network by increasing the retransmission interval with the number of retransmissions. The random jitter factor breaks the time synchronization of multiple message retransmissions, avoiding network congestion caused by the "thunderous herd effect." The execution logic function clarifies the processing rules under different scenarios, ensuring the orderly execution of the retransmission mechanism. This not only improves the success rate of message transmission but also provides timely feedback on unrecoverable transmission anomalies through the alarm trigger mechanism, ensuring the reliability of data interaction.

[0094] In summary, the overall process steps for achieving data interconnection and interoperability based on data connectors are as follows:

[0095] I. Preliminary Preparation Stage

[0096] Business system interface preparation: The business system provides external access interfaces using protocols such as HTTP / HTTPS and AMQP. The optimal protocol method is selected according to business needs, such as FTP / SFTP for large data transmission and AMQP for real-time interaction.

[0097] Access connector assembly: The access connector consists of an information exchange model and a protocol adapter. The information exchange model standardizes and encapsulates the protocol interfaces of the business system, abstractly describing business data assets such as information source, information format, exchange protocol, and path selection address. The protocol adapter converts different protocols, transforming heterogeneous protocols such as HTTP / HTTPS, AMQP, SOAP, and FTP / SFTP into a unified standard protocol.

[0098] II. Registration and Release Phase

[0099] Data catalog registration: The business system reports the exchanged business data catalog information to the regional functional node and obtains a unique identifier; defines system protocol interface parameters, binds the access connector and initiates registration; after the data catalog is published by the business functional node, it is synchronized to the regional and global functional nodes, automatically generates service interface numbers, and forms a complete data catalog. The data catalog contains information format, protocol and content sample.

[0100] Access connector registration: Access connectors, including interfaces, protocol adapters, and information exchange models, complete registration through regional functional nodes, generating an authorization license identification code as a service identifier suffix; registration parameters are initialized when the message flow engine starts and remain resident in memory.

[0101] Authorized service configuration: After the access connector encapsulates the business data service, it is audited through security policies such as privacy computing to form a standard business data service interface; the interface is published in the business function node management interface to provide authorized access to the outside world. Only users / systems that have passed identity verification and permission review can call the interface.

[0102] API service publishing: Assign access permissions and subscription permissions to registered services on the data directory; publish the service protocol access address, such as the HTTP interface address "http: / / xxx.xxx.xxx / Service", through business function nodes to form an API privacy authorization interface data directory resource pool.

[0103] III. Data Interaction Phase

[0104] Data request initiation: Browse the data catalog through the authenticated third-party business system to obtain a brief introduction of the data products; and initiate a data request based on the calling rules, message standards, and protocol types.

[0105] Message encapsulation: The request and response data between the data requester / subscriber and the provider / publisher are encapsulated by the information exchange model to form a message in XML or JSON format. The message structure conforms to Appendix A, including service identifier fields such as SourceSysID and ServiceID, service data fields such as Request and Response, and response information fields such as Status and Code.

[0106] Message queue processing: Message messages flow through the message queue. The message queue serves as a buffer for information exchange between business systems, handles data calls for different service request protocols, and stores received and processed messages according to the access connector rules.

[0107] Message stream processing:

[0108] The message flow creates and executes the computational processing logic of the message flow nodes; constructs the information exchange channel between the application systems at both ends of the bus; reads the information assembly instance data according to the behavior information defined by the access connector; and records the log data generated by the execution operation logic of the message body.

[0109] The message flow engine transmits and processes message messages according to the logical order defined by the message flow; it operates and processes message data in real time; it performs calculations and analyses based on business requirements; it saves the analysis and processing results and records logs; and it distributes messages to target functional units, such as business function nodes and regional function nodes, to achieve rule-based transmission.

[0110] Core competencies working together:

[0111] Data publishing and discovery capabilities: Supports multi-condition combined queries and fuzzy searches in the directory, and displays data in paginated order by relevance and update time; ensures directory synchronization through three synchronization modes: scheduled, real-time, and incremental, and has synchronization status monitoring and anomaly alerts; uses privacy computing encryption algorithms to encrypt sensitive data directories, desensitizes and anonymizes privacy information, and sets fine-grained access control rules; flexibly configures data directory publishing permissions, accurately assigns operation permissions, and records permission operation logs.

[0112] Data interconnection and interoperability capabilities: Access permissions are configured based on the access connector registration information and data sensitivity; the identity and permissions of the requester are verified in real time to block unauthorized or cross-permission access; multiple heterogeneous protocols are automatically identified and converted; semantic conversion is achieved through preset mapping rules or custom mapping tables; the source data structure is analyzed and the data is reorganized according to the requirements of the target system; key information logs are automatically collected and stored in regional functional nodes throughout the entire data interaction process.

[0113] Spatial digital identity capabilities: Employs national cryptographic multi-layer encryption algorithms, identity authentication protocols, and dynamic key management mechanisms to ensure identity security; utilizes national cryptographic algorithms to achieve encrypted data transmission and storage, de-identification, and fine-grained access control, and regularly updates keys; conducts security assessments and access control for application access, monitors data calls and operation behaviors during application operation in real time, triggers warnings and blocks in case of anomalies; deploys optimized SSL / TLS transmission encryption protocols and IPsec secure tunnel technology, monitors transmission link status 24 hours a day, and activates defense mechanisms in case of anomalies.

[0114] Data exchange model: Based on the standardized structure of the identifier field (Header) and the data field (Body), the identifier field contains information such as the model's own identifier and transmission identifier. The data field is divided into pipeline data and business data, recording request pipeline information and exchanged data content. The component manager gives business data service-oriented information communication capabilities.

[0115] Routing and Transmission: When a message passes through the message flow pipeline, the message body content is parsed and matched with the registration data to obtain routing information and transmit it to the destination. Business function nodes directly interface with the business system, responsible for the transmission of business data, log collection, and basic security verification. When an invalid data connector is detected, the transmission is blocked and relevant information is reported to the regional function nodes. After summarizing and analyzing the reported information, the regional function nodes report the business information and monitoring information to the global function nodes, while managing the registration, directory synchronization, and permission allocation of data connectors within their respective regions. The global function nodes manage all regional function nodes in a coordinated manner, control and schedule data connector transaction actions, and maintain the global data directory and security policies.

[0116] IV. Anomaly Handling and Protection Phase

[0117] Message resend mechanism:

[0118] Triggering condition: The message queue manager determines in real time whether a message sending status is failed or timed out and the number of resends has been reached. When, trigger the first If the message is not sent again, the resending process will be terminated.

[0119] Retransmission interval: An exponential backoff combined with a random jitter strategy is adopted, and the interval formula is as follows: , The initial retransmission interval is determined by the message queue manager. Adjust dynamically based on network load. α is the random jitter factor (α∈[0,0.1]).

[0120] Execution and Termination Logic: The message queue manager performs retransmission and updates based on triggering conditions. Value; if sent successfully, i.e. and Terminate retransmission and record the result; if and Terminate retransmission and trigger alarm ( The message is marked as a failure pending manual processing.

[0121] Message Queue Manager Oversight: Oversees instantiated queues, fixes scheduling anomalies, and retransmits data; creates new queues or reclaims idle queue resources based on business needs to handle sudden data concurrency.

[0122] Security Audit and Traceability: By leveraging the full-process logs collected through data interconnectivity and the security protection measures of spatial digital identity capabilities, the entire process of data interaction can be traced. Auditors can quickly locate abnormal operations and improve the efficiency of security incident investigation.

[0123] This embodiment also provides an access connector registration management process, as detailed below:

[0124] Registration Process: The access connector includes interfaces, protocol adapters, and information exchange models. Registration is completed through regional functional nodes, generating an authorization license identification code, which is used as a service identifier suffix. Registration parameters are initialized when the message flow engine starts and reside in memory. When a message flows through the message flow pipeline, the system parses the message body content and matches it with the registration data to obtain routing information and transmit the message to the designated destination.

[0125] Key steps

[0126] Business system interface preparation: The business system provides external access interfaces using protocols such as HTTP / HTTPS and AMQP. The optimal protocol method is selected according to the business scenario. For big data transmission scenarios, FTP / SFTP protocol is selected, and for real-time interaction scenarios, AMQP protocol is selected.

[0127] Data catalog registration: The business system reports the business data catalog information to be exchanged to the regional functional node and obtains a unique identifier; defines the system protocol interface parameters, binds the access connector and initiates registration; after the data catalog is published by the business functional node, it is synchronized to the regional functional node and the global functional node, and the system automatically generates a service interface number to form a complete data catalog containing information format, protocol and content examples.

[0128] API service publishing: Assign access permissions and subscription permissions to registered services in the data directory; publish the service to the outside world through the protocol access address of the service provided by the business function node, such as the HTTP interface address http: / / xxx.xxx.xxx / Service, forming an API privacy authorization interface data directory resource pool.

[0129] Service Invocation: After authentication, the third-party business system browses the data catalog and obtains the data product introduction; according to the invocation rules, message standards and protocol types, it communicates with the bound business system to complete the exchange of data and instructions.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for achieving data interconnection and interoperability based on data connectors, characterized in that, Includes the following steps: S1. The business system provides an external access interface for at least one of the following protocols: HTTP / HTTPS, AMQP, FTP / SFTP, and SOAP, and selects the optimal protocol method; an access connector is constructed consisting of an information exchange model and a protocol adapter. The information exchange model standardizes and encapsulates the business system's protocol interfaces and abstracts and describes business data assets. The protocol adapter converts heterogeneous protocols into a unified standard protocol. S2. The business system reports the data catalog information to the regional functional node and binds the access connector to initiate registration. After the data catalog is published by the business functional node, it is synchronized to the regional and global functional nodes to form a complete data catalog. The access connector completes registration and generates an authorization license identification code through the regional functional node. The registration parameters are initialized and resident in memory when the message flow engine starts. The access connector encapsulates business data services and forms standard interfaces after privacy computation auditing. Access and subscription permissions are allocated and published to the outside world, forming an API privacy authorization interface data catalog resource pool. S3: After authentication, third-party business systems browse the data directory and initiate data requests. The request and response data are encapsulated into XML or JSON format message messages through the information exchange model, flow through the message queue, and are distributed to the target functional unit after being processed by the message stream. Relying on the four core capabilities of data publishing and discovery, data interconnection, spatial digital identity, and data exchange model, directory query, protocol and semantic and structural conversion, full-process security protection, and standardized data transmission are completed. Routing and data transmission are realized through three-layer functional nodes of business, region, and global domain. S4. The message retransmission mechanism is executed through the message queue manager. When a message fails to be sent or times out and the maximum number of retransmissions has not been reached, it is retransmitted using an exponential backoff combined with a random jitter strategy. If the message still fails after reaching the maximum number of retransmissions, an alarm is triggered. The message queue manager monitors queue instances and dynamically adjusts queue resources. Data interaction traceability and abnormal operation location are achieved by using full-process logs and security protection measures.

2. The data interconnection and interoperability implementation method based on a data connector according to claim 1, characterized in that, In S1, the FTP / SFTP protocol is selected for large data transmission scenarios, and the AMQP protocol is selected for real-time interaction scenarios; the abstract description of the business data assets includes information source, information format, exchange protocol, and path selection address.

3. The data interconnection and interoperability implementation method based on a data connector according to claim 1, characterized in that, The complete data catalog described in S2 includes data name, data type, business system to which it belongs, update time, sensitivity level, information format, protocol, and content example; the privacy computation adopts a federated learning algorithm.

4. The data interconnection and interoperability implementation method based on a data connector according to claim 3, characterized in that, The message message described in S3 includes a service identifier field, a service data field, and a response information field. The service identifier field contains SourceSysID and ServiceID, and the response information field contains Status and Code. The message structure conforms to the preset message standard.

5. The data interconnection and interoperability implementation method based on a data connector according to claim 3, characterized in that, The data publishing and discovery capabilities described in S3 support multi-condition combination queries and fuzzy searches based on keywords, categories, and time ranges. Directory synchronization adopts timed, real-time, or incremental modes and has synchronization status monitoring and abnormal alarm functions. The sensitive data directory employs privacy-preserving computation encryption algorithms such as federated learning encryption and homomorphic encryption to desensitize and anonymize private information, sets fine-grained access control rules, and records permission operation logs.

6. The data interconnection and interoperability implementation method based on a data connector according to claim 3, characterized in that, The data interconnection capability described in S3 configures access permissions based on the access connector registration information and data sensitivity, verifies the identity and permissions of the requester in real time, and blocks unauthorized or cross-permission access; automatically identifies heterogeneous protocols such as HTTP / HTTPS, AMQP, SOAP, FTP / SFTP and completes the conversion, realizes semantic conversion through preset mapping rules or custom mapping tables, and reorganizes the source data structure according to the requirements of the target system; and automatically collects key information logs throughout the data interaction process and stores them in the regional functional nodes.

7. The data interconnection and interoperability implementation method based on a data connector according to claim 1, characterized in that, The spatial digital identity capability described in S3 employs multiple encryption algorithms, such as SM4 symmetric encryption and SM2 asymmetric encryption, in accordance with Chinese national cryptographic standards, and combines OAuth2.0, SAML and other identity authentication protocols and dynamic key management mechanisms to ensure identity security. Data transmission uses SM4 encryption, and storage uses SM3 hash encryption. Privacy information is desensitized and field-level fine-grained access control is implemented, and keys are updated regularly. Security assessments and access controls such as vulnerability scanning and code auditing are performed on application access. Data call and operation behavior are monitored in real time, and warnings are triggered and blocked when anomalies occur. Deploy SSL / TLS transmission encryption protocols and IPsec secure tunnel technology based on national cryptographic algorithms, monitor the transmission link status 24 hours a day, and activate defense mechanisms when anomalies occur.

8. The data interconnection and interoperability implementation method based on a data connector according to claim 1, characterized in that, The data exchange model described in S3 consists of a catalog list and a component manager, and its structure is divided into an identifier field and a data field. The catalog list records asset metadata such as data name and data type. The component manager enables business data to communicate information in a service-oriented manner. The identifier field contains information such as model version number and transmission priority. The data field is divided into stream data that records request flow information and business data that records the content of exchanged data.

9. A method for achieving data interconnection and interoperability based on a data connector according to claim 7, characterized in that, The routing and transmission process described in S3 is as follows: after the message is parsed through the message flow pipeline, it is matched with the registration data to calculate and obtain routing information; the business function node connects to the business system and is responsible for data transmission, log collection and basic security verification. When an illegal data connector is detected, the transmission is blocked and the information is reported to the regional function node. After summarizing and analyzing the information, the regional functional nodes report the information to the global functional nodes. At the same time, they manage the registration of data connectors, directory synchronization and permission allocation within their respective regions. The global functional nodes manage all regional functional nodes in a coordinated manner, control and schedule data connector transaction actions, and maintain the global data directory and security policies.

10. A method for achieving data interconnection and interoperability based on a data connector according to claim 7, characterized in that, The message retransmission mechanism described in S4 includes variable definitions, triggering condition functions, retransmission interval formulas, and execution logic functions, as detailed below: (1) Variable definition: The service message to be transmitted is a business data message in XML or JSON format; Message queue manager, responsible for message scheduling and retransmission control; :information The sending status, This indicates that the message was sent successfully. This indicates that the transmission failed. Indicates a timeout; Message sending timeout threshold, determined by Configure according to business scenarios; :information Time elapsed from sending to receiving the response; :information The number of retransmissions, initial value , ; Maximum number of retransmissions, determined by... Dynamic configuration to avoid resource consumption; : No. Second response and the first The time interval between each send / retransmission; Initial retransmission interval, determined by Adjust dynamically based on network load; Random jitter factor This is used to avoid the thunderous herd effect of multiple message retransmissions; :information Alarm trigger status, This indicates that an alarm has been triggered. This indicates that no alarm will be triggered; (2) Triggering condition function: ； in, Indicates triggering the first Second resend This indicates that resending has been terminated; (3) Retransmission interval formula: in, This reflects an index retreat. It exhibits random jitter; (4) Execute the logic function: 。

Images