Multi-subject heterogeneous data exchange method and device, electronic equipment and storage medium

By establishing a hyper-converged cloud platform and multi-subject heterogeneous data exchange method in the multi-subject collaborative interoperability logistics support information system, the problem of data integration between systems is solved, and efficient aggregation and exchange of multi-source heterogeneous data is realized, and multiple sources of heterogeneous data are supported. This data access and encoding standards have been generated to generate a shared exchange platform.

CN118964317AInactive Publication Date: 2024-11-15INFORMATION CENT OF THE LOGISTICS SUPPORT DEPT OF THE CENT MILITARY COMMISSION
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410955427.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the multi-subject collaborative and interoperable logistics support information system, due to the independent research and development of each business information system and the inconsistent technical system, the data encoding standards are different, resulting in difficulty in integrating, aggregating and exchanging data between systems.

Method used

Using multi-subject heterogeneous data exchange method, a hyper-converged cloud platform is established on a network formed by multiple subjects, and a cloud infrastructure module, data resource module, data exchange module, resource directory module, data transmission module and operation and maintenance management and control support module are used. , realizes data aggregation and exchange, uses star topology structure and streaming processing technology to automatically map metadata, and supports a variety of data access methods and encoding standards.

Benefits of technology

The online incremental extraction and high concurrent cleaning conversion of multi-source heterogeneous data is realized, and the purpose of aggregation and exchange of multi-subject heterogeneous data is achieved. A multi-subject collaborative interoperability and sharing exchange platform is generated, supporting spatial database of data files and Metadata maintenance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964317A_ABST
    Figure CN118964317A_ABST
Patent Text Reader

Abstract

The invention relates to a multi-body heterogeneous data exchange method and device, electronic equipment and a storage medium. The multi-agent heterogeneous data exchange method comprises the steps that a star topological structure is adopted, the star topological structure comprises a data exchange center and a plurality of application systems, the data exchange center comprises a distributed streaming processing cluster, and each application system comprises a data exchange agent; the data exchange center is a unified calculation center for data exchange and realizes automatic mapping of metadata; and the data exchange agent cooperates with the data exchange center to complete the functions of metadata extraction, loading and storage. The multi-body heterogeneous data exchange device comprises a cloud infrastructure module, a data resource module, a data exchange module, a resource directory module, a data transmission channel module, an application support module and an operation and maintenance management and control support module. According to the multi-main-body heterogeneous data exchange method provided by the invention, a data exchange general component can be generated, and then a multi-main-body collaborative intercommunication sharing exchange platform is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data exchange technology, and in particular to a method, device, electronic device and storage medium for multi-agent heterogeneous data exchange for multi-agent collaborative intercommunication of logistics support information. Background Art

[0002] Since the business information systems involved in multi-agent collaborative intercommunication are generally independently developed, the technical systems implemented by the systems are not unified, and the data coding standards are also different. The multiple subjects, multiple technical systems, multiple data coding standards, and multiple data types of the data in the multi-agent collaborative intercommunication logistics support information system make data integration, aggregation, and exchange between systems extremely difficult.

[0003] How to quickly and in real time collect or integrate data from existing heterogeneous business systems into a centralized data center environment to support the comprehensive integration of logistics business information systems and big data sharing and analysis applications has become a key issue restricting the comprehensive integration of multi-agent collaborative and interoperable logistics support information systems.

[0004] To this end, in order to address the application scenarios of concurrent, incremental, and automatic real-time data collection from multiple heterogeneous source ends to multiple target libraries, there is an urgent need for an integrated tool and technology for multi-source heterogeneous data collection, conversion, and loading for distributed application fields. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the technical systems for implementing data systems with multiple heterogeneous sources are not unified, and the data encoding standards are also different. The multiple data subjects, multiple technical systems, multiple data encoding standards, and multiple data types in the multi-subject collaborative and interoperable information system lead to difficulties in data integration, aggregation, and exchange between systems.

[0006] In order to solve the above technical problems, according to one aspect of the present invention, a method for multi-subject heterogeneous data exchange is provided, which is used for data aggregation and exchange of multiple subjects, multiple technical systems, multiple data encoding standards, and multiple data types in a multi-subject collaborative information system. The method is implemented based on a device for multi-subject heterogeneous data exchange, and the device includes: a cloud infrastructure module, a data resource module, a data exchange module, a resource directory module, a data transmission channel module, an application support module, and an operation and maintenance control support module. The cloud infrastructure module establishes a hyper-converged cloud platform on multiple networks formed by multiple subjects, and the hyper-converged cloud platform includes three 2U domestic servers, a 10 Gigabit switch, and a 1 Gigabit switch; the data resource module The block includes an exchange library, a subject library and a shared library, wherein the exchange library is a front-end library, which is used to store data collected through the front-end system, the subject library is used to classify the data in the front-end library by subject, and the shared library is used to classify the data in the front-end library by data resource pedigree; the data exchange module includes a front-end system, shared data management, exchange management and data governance, wherein the front-end system is used to store data collection in the front-end library, shared data management is used to standardize the collected data, exchange management is used for data exchange based on stream processing, data exchange based on front-end agent adapters and data exchange based on publish / subscribe, and data governance is used for data screening; the resource directory module is implemented based on the data resource pedigree, and the data resource pedigree adopts metadata. The data describes the characteristics of information resources to form a unified and standardized directory content, and forms a directory information library through effective organization and management of the directory content; the data transmission module is used for message transmission, stream transmission and interface transmission, among which the data transmission provides data transmission means including TCP / UPD efficient transmission framework based on Netty, Socket transmission framework, Rest form data interface service, Http protocol-based data interface service and Kafka-based asynchronous message format; the method of multi-subject heterogeneous data exchange includes: cross-domain data sharing, sharing data in the shared library, introducing an automatic generation system, randomly generating characters of different uppercase and lowercase letters and numbers, and using the RSA (asymmetric encryption algorithm) algorithm to encrypt the public key The data is generated by the data exchange module and the private key, and then converted into XML format for local storage and encrypted by the public key. When the component has a data download demand, the ciphertext data can be downloaded directly through the platform, and then the private key is used to decrypt the data to obtain the decrypted data. When the data exchange module exchanges data, a star topology is adopted. The star topology includes a data exchange center and multiple application systems, wherein the data exchange center includes a distributed streaming processing cluster, and each application system includes a data exchange agent. The data exchange center is a unified computing center for data exchange, which realizes automatic mapping of metadata. The data exchange agent cooperates with the data exchange center to complete functions including metadata extraction, loading, and storage.Data exchange includes data access and data output. When the data exchange module performs data access, it provides access methods for various types of data to the data source. ;

[0007] According to an embodiment of the present invention, metadata information may be transmitted between the data exchange center and the application system in XML format.

[0008] According to an embodiment of the present invention, the data exchange center may use stream processing technology to complete semantic similarity calculation and realize automatic mapping of metadata.

[0009] According to an embodiment of the present invention, the access methods may include ETL (Extract, Transform, Load) customized data capture strategy mode access, front-end library access mode, log incremental capture mode, API service interface mode, and Datagram mode access.

[0010] According to a second aspect of the present invention, there is provided a device for heterogeneous data exchange among multiple subjects, multiple technical systems, multiple data encoding standards, and multiple data types in a multi-subject collaborative information system, and data aggregation and exchange is performed. The device for heterogeneous data exchange among multiple subjects comprises: a cloud infrastructure module, a data resource module, a data exchange module, a resource directory module, a data transmission channel module, an application support module, and an operation and maintenance control support module, wherein the cloud infrastructure module establishes a hyper-converged cloud platform on multiple networks formed by multiple subjects, and the hyper-converged cloud platform comprises three 2U domestic servers, a 10 Gigabit switch, and a 1 Gigabit switch; the data resource module comprises an exchange library, a subject library, and a shared library, wherein the exchange library It is a front-end library, the exchange library is used to store data collected by the front-end system, the subject library is used to classify the data in the front-end library by subject, and the shared library is used to classify the data in the front-end library by data resource pedigree; the data exchange module includes the front-end system, shared data management, exchange management and data governance, among which the front-end system is used to store data collection in the front-end library, shared data management is used to standardize the collected data, exchange management is used for data exchange based on stream processing, data exchange based on front-end agent adapter and data exchange based on publish / subscribe, and data governance is used for data screening; the resource directory module is implemented based on the data resource pedigree, and the data resource pedigree uses metadata to describe the characteristics of information resources to form a unified rule The directory content of the standard is formed through the effective organization and management of the directory content to form a directory information library; the data transmission module is used for message transmission, stream transmission and interface transmission, among which the data transmission provides data transmission means including TCP / UPD efficient transmission framework based on Netty, Socket transmission framework, Rest form data interface service, Http protocol-based data interface service and Kafka-based asynchronous message format; among which, the data exchange module realizes multi-subject heterogeneous data exchange, cross-domain data sharing, data access and data output. Cross-domain data sharing is used to share data in the shared library, introduces an automatic generation system, randomly generates characters of different uppercase and lowercase letters and numbers, and uses the RSA algorithm to perform public key and The private key is generated, converted into XML format and stored locally, encrypted by the public key. When the component has a data download demand, the ciphertext data can be downloaded directly through the platform, and then decrypted by the private key to obtain the decrypted data; when the data exchange module exchanges data, a star topology is adopted. The star topology includes a data exchange center and multiple application systems, wherein the data exchange center includes a distributed streaming processing cluster, and each application system includes a data exchange agent (Agent); the data exchange center is a unified computing center for data exchange, realizing automatic mapping of metadata; the data exchange agent (Agent) cooperates with the data exchange center to complete functions including metadata extraction, loading, and storage;Data exchange includes data access and data output. When the data exchange module performs data access, it provides access methods for various types of data to the data source. ;

[0011] According to an embodiment of the present invention, metadata information may be transmitted between the data exchange center and the application system in XML format.

[0012] According to an embodiment of the present invention, the data exchange center may use stream processing technology to complete semantic similarity calculation and realize automatic mapping of metadata.

[0013] According to an embodiment of the present invention, the access methods may include ETL (Extract, Transform, Load), customized data capture strategy mode access, front-end library access mode, log incremental capture mode, API service interface mode, and Datagram mode access.

[0014] According to a third aspect of the present invention, there is provided an electronic device comprising: a memory, a processor, and a multi-subject heterogeneous data exchange program stored in the memory and executable on the processor, wherein the multi-subject heterogeneous data exchange program implements the steps of the multi-subject heterogeneous data exchange method described above when executed by the processor.

[0015] According to a fourth aspect of the present invention, a computer storage medium is provided, wherein a multi-subject heterogeneous data exchange program is stored on the computer storage medium, and when the multi-subject heterogeneous data exchange program is executed by a processor, the steps of the multi-subject heterogeneous data exchange method described above are implemented.

[0016] Compared with the prior art, the technical solution provided by the embodiments of the present invention can at least achieve the following beneficial effects:

[0017] The present invention integrates technical frameworks such as big data concurrent stream processing, microservice computing, message queues, and distributed message collaborative processing to achieve online incremental extraction and high-concurrency cleaning and conversion of multi-source heterogeneous data, thereby achieving the purpose of multi-subject heterogeneous data aggregation and exchange.

[0018] According to the method for multi-subject heterogeneous data exchange of the present invention, a multi-subject heterogeneous data convergence and exchange technology is provided, a series of general components such as data exchange components are generated, and then a multi-subject collaborative intercommunication shared exchange platform is realized, such as a subject collaborative intercommunication logistics support information shared exchange platform.

[0019] According to the method of multi-subject heterogeneous data exchange of the present invention, multiple spatiotemporal data files can be uploaded online by utilizing multi-source heterogeneous data aggregation and exchange. After successful uploading, the storage and data service registration of the uploaded files can be completed, and the spatialization and storage of the data files can be automatically completed. After the storage is completed, the list of data to be released is automatically entered, and the aggregation dynamics are updated at the same time. It supports the retrieval of document information services in service management and the maintenance of document information service metadata. The data tables in the database under the same network environment are aggregated through the platform, and the setting of restriction information such as the fields opened by the data service is supported. After the aggregation of the registered database service is completed, the data publishing process is automatically entered, and the aggregation dynamics are updated at the same time. It supports the database aggregation personnel to maintain and manage the aggregated data connection information. The stateless services that comply with the HTTP protocol are aggregated online. For GIS services and Rest services that comply with the OGC specification, it supports automatic identification of service-related information and filling in metadata. At the same time, it supports the online registration and management of extended third-party services. After the aggregation is successful, it supports entering the data publishing process link and updating the aggregation dynamics at the same time.

[0020] The present invention adopts a multi-source data fusion technology with fusion conversion as the main method to complete various inquiries such as attribute condition input. While developing a multi-subject heterogeneous data convergence and exchange technology, a multi-source data fusion technology with fusion conversion as the main method is adopted to develop an attribute condition input plug-in, a spatial condition input plug-in, an overlay analysis input plug-in, a buffer analysis input plug-in, and a spatial data output plug-in, etc., respectively realizing the extraction and conversion of data according to different conditions for spatial data sets, and supporting the combined use of various plug-ins. Provides an attribute table input plug-in and an attribute table output plug-in for system users to realize the extraction and conversion of data from non-spatial databases, and supports the combined use of attribute table data with spatial data plug-ins according to conditions. Metadata information is transmitted in XML format, and Excel file input plug-ins, Excel data output plug-ins, GIS file input plug-ins, GIS file output plug-ins, etc. are provided to realize the extraction, conversion and output of data in Excel format and GIS file format. Provides a field selection plug-in, a duplicate record removal plug-in, a value mapping plug-in, a record association plug-in, a record merging plug-in, a sorting and merging plug-in, etc., for realizing the association or intermediate processing of data when integrating and converting the results between different plug-ins. The system automatically checks and converts the collected data fields, data content and other information. Provides a process definition functional area for extracting and converting data into process definitions using different plug-ins according to business needs, and supports storage, updating, and deletion of defined processes, facilitating program calls and automated execution of tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present invention, but are not intended to limit the present invention.

[0022] Figure 1 It is a logical structure diagram showing a method for multi-agent heterogeneous data exchange according to an embodiment of the present invention.

[0023] Figure 2 It is a diagram showing the overall architecture of a multi-agent collaborative and intercommunication logistics support information sharing and exchange platform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Unless otherwise defined, the technical or scientific terms used herein shall have the common meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the patent application specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "a" do not indicate a quantity limitation, but indicate the existence of at least one.

[0026] Figure 1 is a logical structure diagram showing a method for multi-agent heterogeneous data exchange according to an embodiment of the present invention; Figure 2 It is a diagram showing the overall architecture of a multi-agent collaborative and intercommunication logistics support information sharing and exchange platform according to an embodiment of the present invention.

[0027] like Figure 1 and Figure 2 As shown, the method for multi-subject heterogeneous data exchange is used for data aggregation and exchange of multiple subjects, multiple technical systems, multiple data encoding standards, and multiple data types in a multi-subject collaborative and intercommunication information system. The method is implemented based on a device for multi-subject heterogeneous data exchange, and the device includes: a cloud infrastructure module, a data resource module, a data exchange module, a resource directory module, a data transmission channel module, an application support module, and an operation and maintenance control support module.

[0028] Among them, the cloud infrastructure module establishes a hyper-converged cloud platform on multiple networks formed by multiple entities. The hyper-converged cloud platform includes three 2U domestic servers, a 10 Gigabit switch and a Gigabit switch.

[0029] The data resource module includes an exchange library, a subject library and a shared library, wherein the exchange library is a front-end library used to store data collected through the front-end system, the subject library is used to classify the data in the front-end library by subject, and the shared library is used to classify the data in the front-end library by data resource lineage.

[0030] The data exchange module includes the front-end system, shared data management, exchange management and data governance. The front-end system is used to store the collected data in the front-end library, the shared data management is used to standardize the collected data, the exchange management is used for data exchange based on stream processing, data exchange based on the front-end agent adapter and data exchange based on publish / subscribe, and the data governance is used for data screening.

[0031] The resource directory module is implemented based on the data resource genealogy. The data resource genealogy uses metadata to describe the characteristics of information resources to form a unified and standardized directory content. Through the effective organization and management of the directory content, a directory information library is formed.

[0032] The data transmission module is used for message transmission, stream transmission and interface transmission. The data transmission provides data transmission means including TCP / UPD efficient transmission framework based on Netty, Socket transmission framework, Rest form of data interface service, Http protocol-based data interface service and Kafka-based asynchronous message format.

[0033] Methods for multi-agent heterogeneous data exchange include:

[0034] Cross-domain data sharing: sharing data in the shared library, introducing an automatic generation system, randomly generating characters of different uppercase and lowercase letters and numbers, generating public and private keys through the RSA algorithm, converting them into XML format for local storage, and encrypting them with the public key. When the component has a data download demand, it can directly download the ciphertext data through the platform, and then decrypt it with the private key to obtain the decrypted data;

[0035] When the data exchange module exchanges data, it adopts a star topology structure, which includes a data exchange center and multiple application systems. The data exchange center includes a distributed stream processing cluster, and each application system includes a data exchange agent. The data exchange center is a unified computing center for data exchange, which realizes automatic mapping of metadata. The data exchange agent cooperates with the data exchange center to complete functions including metadata extraction, loading, and storage.

[0036] Data exchange includes data access and data output. When performing data access, the data exchange module provides access methods for various types of data to the data source.

[0037] The present invention integrates technical frameworks such as big data concurrent stream processing, microservice computing, message queues, and distributed message collaborative processing to achieve online incremental extraction and high-concurrency cleaning and conversion of multi-source heterogeneous data, thereby achieving the purpose of multi-subject heterogeneous data aggregation and exchange.

[0038] According to one or some embodiments of the present invention, metadata information is transmitted between the data exchange center and the application system in XML format.

[0039] According to one or some embodiments of the present invention, the data exchange center uses streaming processing technology to complete semantic similarity calculation and realize automatic mapping of metadata.

[0040] According to one or some embodiments of the present invention, the access methods include ETL (Extract, Transform, Load), customized data capture strategy mode access, front-end library access mode, log incremental capture mode, API service interface mode, and Datagram mode access.

[0041] According to the method of multi-subject heterogeneous data exchange of the present invention, multiple spatiotemporal data files can be uploaded online by utilizing multi-source heterogeneous data aggregation and exchange. After successful uploading, the storage and data service registration of the uploaded files can be completed, and the spatialization and storage of the data files can be automatically completed. After the storage is completed, the list of data to be released is automatically entered, and the aggregation dynamics are updated at the same time. It supports the retrieval of document information services in service management and the maintenance of document information service metadata. The data tables in the database under the same network environment are aggregated through the platform, and it supports setting restriction information such as fields opened by data services. After the aggregation of registered database services is completed, the data publishing process is automatically entered, and the aggregation dynamics are updated at the same time. It supports the database aggregation personnel to maintain and manage the aggregated data connection information. The stateless services that comply with the HTTP protocol are aggregated online. For GIS services and Rest services that comply with the OGC specification, it supports automatic identification of service-related information and filling in metadata. At the same time, it supports the online registration and management of extended third-party services. After the aggregation is successful, it supports entering the data publishing process link and updating the aggregation dynamics at the same time.

[0042] According to a second aspect of the present invention, there is provided a device for multi-subject heterogeneous data exchange, which is used for data aggregation and exchange among multiple subjects, multiple technical systems, multiple data encoding standards, and multiple data types in a multi-subject collaborative intercommunication information system.

[0043] The device for multi-subject heterogeneous data exchange includes: cloud infrastructure module, data resource module, data exchange module, resource directory module, data transmission channel module, application support module and operation and maintenance management support module.

[0044] Among them, the cloud infrastructure module establishes a hyper-converged cloud platform on multiple networks formed by multiple entities. The hyper-converged cloud platform includes three 2U domestic servers, a 10 Gigabit switch and a Gigabit switch.

[0045] The data resource module includes an exchange library, a subject library and a shared library, wherein the exchange library is a front-end library used to store data collected through the front-end system, the subject library is used to classify the data in the front-end library by subject, and the shared library is used to classify the data in the front-end library by data resource lineage.

[0046] The data exchange module includes the front-end system, shared data management, exchange management and data governance. The front-end system is used to store the collected data in the front-end library, the shared data management is used to standardize the collected data, the exchange management is used for data exchange based on stream processing, data exchange based on the front-end agent adapter and data exchange based on publish / subscribe, and the data governance is used for data screening.

[0047] The resource directory module is implemented based on the data resource genealogy. The data resource genealogy uses metadata to describe the characteristics of information resources to form a unified and standardized directory content. Through the effective organization and management of the directory content, a directory information library is formed.

[0048] The data transmission module is used for message transmission, stream transmission and interface transmission. The data transmission provides data transmission means including TCP / UPD efficient transmission framework based on Netty, Socket transmission framework, Rest form of data interface service, Http protocol-based data interface service and Kafka-based asynchronous message format.

[0049] Among them, the data exchange module realizes multi-subject heterogeneous data exchange, cross-domain data sharing, data access and data output. Cross-domain data sharing is used to share data in the shared library. An automatic generation system is introduced to randomly generate characters of different uppercase and lowercase letters and numbers. Public and private keys are generated through the RSA algorithm, which are converted into XML format for local storage and encrypted with the public key. When the component has a data download demand, the ciphertext data can be downloaded directly through the platform. At this time, the private key is used to decrypt it to obtain the decrypted data.

[0050] When the data exchange module exchanges data, it adopts a star topology structure, which includes a data exchange center and multiple application systems. The data exchange center includes a distributed streaming processing cluster, and each application system includes a data exchange agent (Agent); the data exchange center is a unified computing center for data exchange, which realizes automatic mapping of metadata; the data exchange agent (Agent) cooperates with the data exchange center to complete functions including metadata extraction, loading, and warehousing.

[0051] Data exchange includes data access and data output. When performing data access, the data exchange module provides access methods for various types of data to the data source.

[0052] The present invention adopts a multi-source data fusion technology with fusion conversion as the main method to complete various inquiries such as attribute condition input. While developing a multi-subject heterogeneous data convergence and exchange technology, a multi-source data fusion technology with fusion conversion as the main method is adopted to develop an attribute condition input plug-in, a spatial condition input plug-in, an overlay analysis input plug-in, a buffer analysis input plug-in, and a spatial data output plug-in, etc., respectively realizing the extraction and conversion of data according to different conditions for spatial data sets, and supporting the combined use of various plug-ins. Provides an attribute table input plug-in and an attribute table output plug-in for system users to realize the extraction and conversion of data from non-spatial databases, and supports the combined use of attribute table data with spatial data plug-ins according to conditions. Provides an Excel file input plug-in, an Excel data output plug-in, a GIS file input plug-in, a GIS file output plug-in, etc., for realizing the extraction, conversion and output of data in Excel format and GIS file format. Provides a field selection plug-in, a duplicate record removal plug-in, a value mapping plug-in, a record association plug-in, a record merging plug-in, a sorting and merging plug-in, etc., for realizing the association or intermediate processing of data when integrating and converting the results between different plug-ins. The system automatically checks and converts the collected data fields, data content and other information. Provides a process definition functional area for extracting and converting data into process definitions using different plug-ins according to business needs, and supports storage, updating, and deletion of defined processes, facilitating program calls and automated execution of tasks.

[0053] According to one or some embodiments of the present invention, metadata information is transmitted between the data exchange center and the application system in XML format.

[0054] According to one or some embodiments of the present invention, the data exchange center uses streaming processing technology to complete semantic similarity calculation and realize automatic mapping of metadata.

[0055] According to one or some embodiments of the present invention, the access methods include ETL (Extract, Transform, Load), customized data capture strategy mode access, front-end library access mode, log incremental capture mode, API service interface mode, and Datagram mode access.

[0056] According to the method for multi-subject heterogeneous data exchange of the present invention, a multi-subject heterogeneous data convergence and exchange technology is provided, a series of general components such as data exchange components are generated, and then a multi-subject collaborative intercommunication shared exchange platform is realized, such as a subject collaborative intercommunication logistics support information shared exchange platform.

[0057] According to another aspect of the present invention, a device for multi-subject heterogeneous data exchange is provided, comprising: a memory, a processor, and a multi-subject heterogeneous data exchange program stored in the memory and executable on the processor, wherein the multi-subject heterogeneous data exchange program implements the steps of the multi-subject heterogeneous data exchange method described above when executed by the processor.

[0058] According to the present invention, a computer storage medium is also provided.

[0059] A multi-subject heterogeneous data exchange program is stored on a computer storage medium, and when the multi-subject heterogeneous data exchange program is executed by a processor, the steps of the multi-subject heterogeneous data exchange method described above are implemented.

[0060] Among them, the method implemented when the multi-agent heterogeneous data exchange program running on the processor is executed can refer to the various embodiments of the multi-agent heterogeneous data exchange method of the present invention, and will not be repeated here.

[0061] The present invention also provides a computer program product.

[0062] The computer program product of the present invention comprises a multi-agent heterogeneous data exchange program, and when the multi-agent heterogeneous data exchange program is executed by a processor, the steps of the multi-agent heterogeneous data exchange method described above are implemented.

[0063] Among them, the method implemented when the multi-agent heterogeneous data exchange program running on the processor is executed can refer to the various embodiments of the multi-agent heterogeneous data exchange method of the present invention, and will not be repeated here.

[0064] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0065] The above description is merely an exemplary embodiment of the present invention and is not intended to limit the protection scope of the present invention. The protection scope of the present invention is determined by the appended claims.

Claims

1. A method for exchanging heterogeneous data among multiple subjects, multiple technical systems, multiple data encoding standards, and multiple data types in a multi-subject collaborative information system, for data aggregation and exchange. The method is implemented based on a device for exchanging heterogeneous data among multiple subjects. The device include: Cloud infrastructure module, data resource module, data exchange module, resource directory module, data transmission channel module, application support module and operation and maintenance control support module. The cloud infrastructure module establishes a hyper-converged cloud platform on multiple networks formed by multiple entities, and the hyper-converged cloud platform includes three 2U domestic servers, a 10G switch and a 1G switch; The data resource module includes an exchange library, a subject library and a shared library, wherein the exchange library is a front-end library, the exchange library is used to store data collected by the front-end system, the subject library is used to classify the data in the front-end library by subject, and the shared library is used to classify the data in the front-end library by data resource pedigree; The data exchange module includes a front-end system, shared data management, exchange management and data governance, wherein the front-end system is used to collect and store data in the front-end library, the shared data management is used to standardize the collected data, the exchange management is used for data exchange based on stream processing, data exchange based on the front-end agent adapter and data exchange based on publish / subscribe, and the data governance is used for data screening; The resource directory module is implemented based on the data resource pedigree, which uses metadata to describe the characteristics of information resources to form a unified and standardized directory content, and forms a directory information library through effective organization and management of the directory content; The data transmission module is used for message transmission, stream transmission and interface transmission, wherein the data transmission provides data transmission means including TCP / UPD efficient transmission framework based on Netty, Socket transmission framework, data interface service in the form of REST, data interface service based on Http protocol and asynchronous message format based on Kafka; The method for exchanging multi-agent heterogeneous data includes: Cross-domain data sharing: sharing data in the shared library, introducing an automatic generation system, randomly generating characters of different uppercase and lowercase letters and numbers, generating public and private keys through the RSA algorithm, converting them into XML format for local storage, and encrypting them with the public key. When the component has a data download demand, it can directly download the ciphertext data through the platform, and then decrypt it with the private key to obtain the decrypted data; When the data exchange module performs data exchange, a star topology is adopted, and the star topology includes a data exchange center and multiple application systems, wherein the data exchange center includes a distributed stream processing cluster, and each application system includes a data exchange agent; the data exchange center is a unified computing center for data exchange, realizing automatic mapping of metadata; the data exchange agent cooperates with the data exchange center to complete functions including metadata extraction, loading, and warehousing; Data exchange includes data access and data output. When performing data access, the data exchange module provides access methods for various types of data to the data source.

2. The method for exchanging heterogeneous data among multiple agents as claimed in claim 1, in, The metadata information is transmitted between the data exchange center and the application system in XML format.

3. The method for exchanging heterogeneous data among multiple agents as claimed in claim 1, in, The data exchange center uses stream processing technology to complete semantic similarity calculation and realize automatic mapping of metadata.

4. The method for exchanging heterogeneous data among multiple agents as claimed in claim 1, in, The access modes include ETL customized data capture strategy mode access, front-end library access mode, log incremental capture mode, API service interface mode, and Datagram mode access.

5. A device for exchanging heterogeneous data among multiple subjects, used for data aggregation and exchange among multiple subjects, multiple technical systems, multiple data encoding standards, and multiple data types in a multi-subject collaborative information system. The device for exchanging heterogeneous data of multiple subjects include: Cloud infrastructure module, data resource module, data exchange module, resource directory module, data transmission channel module, application support module and operation and maintenance control support module. The cloud infrastructure module establishes a hyper-converged cloud platform on multiple networks formed by multiple entities, and the hyper-converged cloud platform includes three 2U domestic servers, a 10G switch and a 1G switch; The data resource module includes an exchange library, a subject library and a shared library, wherein the exchange library is a front-end library, the exchange library is used to store data collected by the front-end system, the subject library is used to classify the data in the front-end library by subject, and the shared library is used to classify the data in the front-end library by data resource pedigree; The data exchange module includes a front-end system, shared data management, exchange management and data governance, wherein the front-end system is used to collect and store data in the front-end library, the shared data management is used to standardize the collected data, the exchange management is used for data exchange based on stream processing, data exchange based on the front-end agent adapter and data exchange based on publish / subscribe, and the data governance is used for data screening; The resource directory module is implemented based on the data resource pedigree, which uses metadata to describe the characteristics of information resources to form a unified and standardized directory content, and forms a directory information library through effective organization and management of the directory content; The data transmission module is used for message transmission, stream transmission and interface transmission, wherein the data transmission provides data transmission means including TCP / UPD efficient transmission framework based on Netty, Socket transmission framework, data interface service in the form of REST, data interface service based on Http protocol and asynchronous message format based on Kafka; The data exchange module realizes multi-subject heterogeneous data exchange, cross-domain data sharing, data access and data output. Cross-domain data sharing is used to share data in a shared library. An automatic generation system is introduced to randomly generate characters of different uppercase and lowercase letters and numbers. Public and private keys are generated through the RSA algorithm, which are converted into XML format for local storage and encrypted by the public key. When a component has a data download demand, the ciphertext data can be downloaded directly through the platform, and then decrypted by the private key to obtain the decrypted data. When the data exchange module performs data exchange, a star topology is adopted, and the star topology includes a data exchange center and multiple application systems, wherein the data exchange center includes a distributed stream processing cluster, and each application system includes a data exchange agent; the data exchange center is a unified computing center for data exchange, realizing automatic mapping of metadata; the data exchange agent cooperates with the data exchange center to complete functions including metadata extraction, loading, and warehousing; Data exchange includes data access and data output. When performing data access, the data exchange module provides access methods for various types of data to the data source.

6. The device for exchanging heterogeneous data among multiple agents as claimed in claim 5, in, The metadata information is transmitted between the data exchange center and the application system in XML format.

7. The device for exchanging heterogeneous data among multiple agents as claimed in claim 5, in, The data exchange center uses stream processing technology to complete semantic similarity calculation and realize automatic mapping of metadata.

8. The device for exchanging heterogeneous data among multiple agents as claimed in claim 5, in, The access modes include ETL customized data capture strategy mode access, front-end library access mode, log incremental capture mode, API service interface mode, and Datagram mode access.

9. An electronic device, include: A memory, a processor, and a multi-agent heterogeneous data exchange program stored in the memory and executable on the processor, wherein the multi-agent heterogeneous data exchange program, when executed by the processor, implements the steps of the multi-agent heterogeneous data exchange method as described in any one of claims 1 to 4.

10. A computer storage medium, in, The computer storage medium stores a multi-agent heterogeneous data exchange program, which, when executed by a processor, implements the steps of the multi-agent heterogeneous data exchange method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • An emergency resource sharing and exchange system

    CA3063117A1

  • Method for constructing government affair big data treatment system

    CN110781236A

  • Cloud native hybrid integration method based on block chain

    CN114840329A

  • Enterprise-level customer information business processing system based on rsa encryption algorithm

    CN117879822A