A method for integrating heterogeneous information data

Abstract

The invention discloses a heterogeneous information data integration method, which relates to the technical field of data integration. A central integration node is configured, a distributed node is created for each of several data sources, all the distributed nodes obtain the node address of the central integration node and establish a connection relationship, a corresponding node deployment environment is created for the distributed nodes that establish the connection relationship, and the heterogeneous information of the data source corresponding to each distributed node is imported into the node deployment environment, and then respective integration request files are generated, and the integration request files of all the distributed nodes are uploaded to the central integration node. The central integration node reviews the received integration request files, and data integration is performed on the reviewed integration request files, and then multi-source integrated data is generated, abnormal integrated data points in the multi-source integrated data are screened out, and data correction is performed on the multi-source integrated data at the abnormal integrated data points.

Description

Technical Field

[0001] The present invention relates to the technical field of data integration, and in particular to a method for integrating heterogeneous information data. Background Art

[0002] With the rapid development of information technology, various information systems have emerged continuously, generating a large amount of heterogeneous information data. Heterogeneous information data refers to data with differences in data format, data structure, data semantics, etc. In the process of data analysis and application, how to effectively integrate these heterogeneous information data has become a problem that needs to be solved urgently.

[0003] Existing data integration methods have the following shortcomings:

[0004] Low integration efficiency: Existing methods often require a lot of time and computing resources to process large amounts of heterogeneous data; Low data quality: Due to the differences between heterogeneous data, the integrated data is prone to problems such as semantic conflicts and data loss. Summary of the invention

[0005] In order to solve the above problems, the purpose of the present invention is to provide a method for integrating heterogeneous information data.

[0006] The object of the present invention can be achieved by the following technical solution: A method for integrating heterogeneous information data, comprising the following steps:

[0007] Step S1: configure the central integration node;

[0008] Step S2: Create a distributed node for each of the multiple data sources, and all the distributed nodes obtain the node address of the central integration node and establish a connection relationship;

[0009] Step S3: creating a corresponding node deployment environment for the distributed nodes that establish a connection relationship, and importing the heterogeneous information of the data source corresponding to each distributed node into the node deployment environment, and then generating respective integration request files;

[0010] Step S4: Upload the integration request files of all distributed nodes to the central integration node;

[0011] Step S5: The central integration node reviews the received integration request files, performs data integration on the reviewed integration request files, and generates multi-source integrated data;

[0012] Step S6: Screen out abnormal integrated data points in the multi-source integrated data, and perform data correction on the multi-source integrated data at the abnormal integrated data points.

[0013] Furthermore, the process of configuring the central integration node includes:

[0014] Create a central integration node, set up the database of the central integration node and configure it. The configuration includes allocating database memory, configuring the database corresponding firewall information, and configuring the database permission access list;

[0015] Create a node address for the central integration node. The node address is a dynamic IP address. The dynamic IP address is associated with a corresponding address list. The address list is used to record the address sequence corresponding to the dynamic IP address in different time periods.

[0016] Configure the communication protocol for the central integration node, and perform node performance test on the central integration node after the communication protocol configuration is completed. The node performance test includes stress test, security test and disaster recovery test. When all node performance tests of the central integration node pass, the central integration node is configured successfully. Otherwise, continue to configure the central integration node until the central integration node is configured successfully.

[0017] Furthermore, a distributed node is created for each of the plurality of data sources, and the process in which all the distributed nodes obtain the node address of the central integration node and establish a connection relationship includes:

[0018] Number several data sources and record them as i, i = 1, 2, 3, ..., n, where n is a natural number greater than 0, create a distributed node for each data source, and bind the source address of the data source to the distributed node, and record the source address bound to the distributed node created by the data source numbered i as Address[i];

[0019] Each distributed node obtains the node address of the central integration node according to the current time period, and establishes its own address mapping sequence pair with the central integration node. The address mapping sequence pair of the distributed node corresponding to the data source numbered i is recorded as G[i], G[i] =<Address[i],Address[Time]> , Address[Time] is the node address corresponding to the central integration node in the corresponding time period;

[0020] When the address mapping sequence pair between the distributed node and the central integration node is established, the distributed node sends a connection establishment request file created by itself to the central integration node. After the central integration node receives the connection establishment request file, it enters its own communication protocol into the connection establishment request file and returns it to the distributed node that sent it. After the distributed node configures the returned communication protocol, it establishes a connection relationship between the distributed node and the central integration node.

[0021] Furthermore, the process of creating a corresponding node deployment environment for the distributed nodes that establish a connection relationship includes:

[0022] Each distributed node that has established a connection relationship is used as a deployment object for creating a node deployment environment. The operation process of creating a node deployment environment corresponding to the deployment object includes node environment construction, node resource configuration and middleware deployment, wherein the node environment construction, node resource configuration and middleware deployment are performed sequentially;

[0023] The content of node environment construction is: build a node environment for each distributed node, initialize the node environment, convert the node environment to a data security state, and set the environment variables corresponding to the node environment;

[0024] The content of node resource configuration is: allocate required node resources to the node environment that has completed environment initialization and setting of environment variables. The node resources include server, running storage space and network bandwidth;

[0025] The content of middleware deployment is: deploying different types of data operation middleware related to data integration for the node environment corresponding to each distributed node, and the different types of data operation middleware include data synchronization middleware, data analysis middleware and data conversion middleware.

[0026] Furthermore, the process of importing the heterogeneous information of the data source corresponding to each distributed node into the node deployment environment and then generating respective integration request files includes:

[0027] Import the heterogeneous information represented by the data source corresponding to each distributed node into the respective node deployment environment. When the heterogeneous information is detected in the node deployment environment, start all data operation middleware deployed in the corresponding node deployment environment.

[0028] The data format of the heterogeneous information in the current node deployment environment is analyzed through the data analysis middleware. If the data format meets the preset final data format required for data integration, no operation is performed. Otherwise, a data conversion identifier is marked for the current heterogeneous information.

[0029] The data conversion middleware receives all heterogeneous information processed by the data analysis middleware, and converts the format of the heterogeneous information marked with data conversion identifiers, thereby generating an initial integration request file for each distributed node, binding the data synchronization middleware with the initial integration request file, and generating a final integration request file.

[0030] Furthermore, the process of uploading the integration request files of all distributed nodes to the central integration node includes:

[0031] Upload the corresponding integration request file of each distributed node to the central integration node, and the data synchronization middleware bound to each integration request file performs real-time data synchronization. Data synchronization is performed according to the preset synchronization frequency, which is changed by the data maintenance personnel according to the actual data synchronization situation;

[0032] Monitor the upload rate of each distributed node uploading the integration request file to the central integration node, and record the upload rate as V 上传 , set the alarm interval corresponding to the upload rate, and record the alarm interval as Ω 告警 , when V 上传 Ω 告警 When the rate abnormality alarm is generated, it will be pushed to the data maintenance personnel, who will adjust the upload rate to make V 上传 Ω 告警 , when V 上传 Ω 告警 No operation is performed when

[0033] Furthermore, the central integration node reviews the received integration request files, performs data integration on the reviewed integration request files, and then generates multi-source integrated data, including:

[0034] The central integration node reviews all received integration request files, including a first review operation and a second review operation, wherein the first review operation includes file verification and content review, and the second review operation includes authority check and conflict detection;

[0035] According to the passing status of the first review operation and the second review operation, different review decisions are executed, and the review decisions include approval decision, rejection decision and feedback decision;

[0036] When both the first review operation and the second review operation are successful, the approval decision is executed, indicating that the review of the corresponding integration request file is passed and the data integration is approved;

[0037] When at least one of the first review operation and the second review operation fails, a rejection decision is executed, indicating that the review of the corresponding integration request file has not passed, and data integration is rejected, and rejection information is given, which is the specific data content that has not passed the review;

[0038] After the first review operation and the second review operation are completed, a feedback decision is executed. The content of the feedback decision is: the review result is fed back to each distributed node, and for the distributed node that executes the rejection decision, a modification suggestion for the corresponding integration request file is provided;

[0039] Data integration is performed on all approved integration request files at the central integration node to generate multi-source comprehensive data.

[0040] Furthermore, the process of screening out abnormal integrated data points in the multi-source integrated data and correcting the multi-source integrated data at the abnormal integrated data points includes:

[0041] The multi-source integrated data is composed of data segments of a plurality of integrated data points, and the data segments of the plurality of integrated data points are numbered and the number is recorded as j, then j=1, 2, 3, ..., m, where m is a natural number greater than 0;

[0042] Starting from j=1, traverse the data segments corresponding to several integrated data points in order from small to large, mark the integrated data points corresponding to the data segments with abnormalities as abnormal integrated data points, and correct the data segments corresponding to the abnormal integrated data points. The anomalies in the data segments include data missing, data duplication, and data errors.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. By configuring the central integration node, a distributed node is created for each of the data sources. Each distributed node establishes a connection relationship with the central integration node. A node deployment environment is created for each of the distributed nodes that establish a connection relationship. The heterogeneous information of the data source corresponding to each distributed node is imported into the node deployment environment, and then the respective integration request files are generated. This enables each data source to prepare for the preliminary integration work in its own distributed node, effectively improving the integration efficiency of subsequent data integration.

[0045] 2. By uploading the integration request files of all distributed nodes to the central integration node, the central integration node will review the received integration request files, perform data integration on the reviewed integration request files, and then generate multi-source comprehensive data, screen out abnormal integration data points in the multi-source comprehensive data, and correct the multi-source comprehensive data at the abnormal integration data points, so as to achieve timely processing of semantic conflicts, data loss, data anomalies and other problems when different heterogeneous information is integrated, thereby ensuring the data quality of the multi-source comprehensive data obtained after data integration. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0047] like Figure 1 As shown, a heterogeneous information data integration method includes the following steps:

[0048] Step S1: configure the central integration node;

[0049] Step S2: Create a distributed node for each of the multiple data sources, and all the distributed nodes obtain the node address of the central integration node and establish a connection relationship;

[0050] Step S3: creating a corresponding node deployment environment for the distributed nodes that establish a connection relationship, and importing the heterogeneous information of the data source corresponding to each distributed node into the node deployment environment, and then generating respective integration request files;

[0051] Step S4: Upload the integration request files of all distributed nodes to the central integration node;

[0052] Step S5: The central integration node reviews the received integration request files, performs data integration on the reviewed integration request files, and generates multi-source integrated data;

[0053] Step S6: Screen out abnormal integrated data points in the multi-source integrated data, and perform data correction on the multi-source integrated data at the abnormal integrated data points.

[0054] It should be further explained that, in the specific implementation process, the process of configuring the central integration node includes:

[0055] Create a central integration node and set up a database for the central integration node. Configure the database of the central integration node, including allocating database memory, configuring database corresponding firewall information, and configuring the database permission access list.

[0056] Creating a node address for the central integration node, the node address is a dynamic IP address, the dynamic IP address is associated with a corresponding address list, the address list is used to record the address sequence corresponding to the dynamic IP address in different time periods;

[0057] Configure the communication protocol for the central integration node, and perform node performance testing on the central integration node after the communication protocol configuration is completed, wherein the node performance testing includes stress testing, security testing, and disaster recovery testing;

[0058] The stress test content is: import a data packet into the central integration node, set the expected processing time of the current data packet, obtain the amount of data to be processed corresponding to the data packet, set the data processing rate of the central integration node, and record the data processing rate and the amount of data to be processed corresponding to the data packet as V 处理 and D 数据包 , and then according to V 处理 and D 数据包 Get the actual processing time of the central integration node for processing the data packet;

[0059] The expected processing time is denoted as T 期望 , the actual processing time is recorded as T 实际 ;

[0060] Then, T 实际 =D 数据包 / V 处理 ;

[0061] When T 实际 >T 期望 When , it means that the central integration node can process the data packets of the current amount of data to be processed as expected, and the stress test passes;

[0062] When T 实际 ≤T 期望 The stress test failed.

[0063] The content of the security test is: injecting an attack file into the central integration node, the attack file is composed of a number of attack information segments, each attack information segment is associated with an attack operation performed on the data stored in the central integration node, and the attack operation includes replacing data, deleting data, and adding data;

[0064] The firewall of the central integration node is used to filter the attack information segments. The total number of attack information segments included in the attack file is recorded as N 总 , the number of attack information segments filtered by the firewall is recorded as N 过滤 ;

[0065] According to N 总 and N 过滤 Get the filtering success rate of the firewall, and record the filtering success rate as τ, then τ=N 过滤 / N 总 , set the success judgment value and record it as Sc. When τ≥Sc, the security test of the central integration node passes. When τ<Sc, the security test fails.

[0066] The content of the disaster recovery test is: simulating a data disaster scenario for the central integration node. In the data disaster scenario, the database associated with the central integration node performs data backup and saves, and determines the difference between the amount of data after the data backup and save and the amount of data before the save. If the difference exceeds the preset upper limit, the disaster recovery test fails, otherwise, the disaster recovery test passes.

[0067] When all node performance tests of the central integration node pass, the central integration node is configured successfully, otherwise, continue to configure the central integration node until the central integration node is configured successfully.

[0068] It should be further explained that, in the specific implementation process, a distributed node is created for each of the multiple data sources, and the process in which all the distributed nodes obtain the node address of the central integration node and establish a connection relationship includes:

[0069] Number several data sources and record the number as i, then i = 1, 2, 3, ..., n, where n is a natural number greater than 0. Create a distributed node for each data source and bind the source address of the corresponding data source to each distributed node. The source address bound to the distributed node created by the data source numbered i is recorded as Address[i].

[0070] Each distributed node obtains the node address of the central integration node according to the current time period, and establishes its own address mapping sequence pair with the central integration node. The address mapping sequence pair of the data source numbered i corresponding to the distributed node is recorded as G[i], then G[i] =<Address[i],Address[Time]> , where Address[Time] is the node address corresponding to the central integration node in the corresponding time period;

[0071] When the address mapping sequence pair between the distributed node and the central integration node is established, the distributed node sends a connection establishment request file created by itself to the central integration node. After the central integration node receives the connection establishment request file, it enters its own communication protocol into the connection establishment request file and returns it to the distributed node that sent it. The distributed node configures the returned communication protocol, and then establishes a connection relationship between the distributed node and the central integration node.

[0072] It should be further explained that, in a specific implementation process, the process of creating a corresponding node deployment environment for the distributed nodes that establish a connection relationship includes:

[0073] Each distributed node that has established a connection relationship is used as a deployment object for creating a node deployment environment. The operation process of creating a node deployment environment corresponding to the deployment object includes node environment construction, node resource configuration and middleware deployment, wherein the node environment construction, node resource configuration and middleware deployment are performed sequentially;

[0074] The content of node environment construction is: build a node environment for each distributed node, initialize the node environment, convert the node environment to a data security state, and set the environment variables corresponding to the node environment;

[0075] The content of node resource configuration is: allocate required node resources to the node environment that has completed environment initialization and setting of environment variables. The node resources include server, running storage space and network bandwidth;

[0076] The content of middleware deployment is: deploying different types of data operation middleware related to data integration for the node environment corresponding to each distributed node, and the different types of data operation middleware include data synchronization middleware, data analysis middleware and data conversion middleware.

[0077] It should be further explained that, in the specific implementation process, the process of importing the heterogeneous information of the data source corresponding to each distributed node into the node deployment environment and then generating the respective integration request files includes:

[0078] When each distributed node successfully creates its own node deployment environment, the heterogeneous information represented by the data source corresponding to each distributed node is imported into the respective node deployment environment. When the heterogeneous information is detected in the node deployment environment, all data operation middleware deployed in the corresponding node deployment environment are started, and the respective heterogeneous information is processed by the data operation middleware. The specific processing content is as follows:

[0079] The data format of the heterogeneous information in the current node deployment environment is analyzed through the data analysis middleware. If the data format meets the preset final data format required for data integration, no operation is performed. Otherwise, a data conversion identifier is marked for the current heterogeneous information.

[0080] The data conversion middleware receives all heterogeneous information processed by the data analysis middleware, and performs format conversion on the heterogeneous information marked with data conversion identifiers, thereby generating an initial integration request file for each distributed node.

[0081] Bind the data synchronization middleware with the initial integration request file to generate the final integration request file.

[0082] It should be further explained that, in the specific implementation process, the process of uploading the integration request files of all distributed nodes to the central integration node includes:

[0083] Upload the integration request file corresponding to each distributed node to the central integration node, and the data synchronization middleware bound to each integration request file performs real-time data synchronization. The data synchronization is performed according to the preset synchronization frequency, which is changed by the data maintenance personnel according to the actual data synchronization situation;

[0084] Monitor the upload rate of each distributed node uploading the integration request file to the central integration node, and record the upload rate as V 上传 , set the alarm interval corresponding to the upload rate, and record the alarm interval as Ω 告警 , when V 上传 Ω 告警When the rate abnormality alarm is generated, it will be pushed to the data maintenance personnel, who will adjust the upload rate to make V 上传 Ω 告警 , when V 上传 Ω 告警 No operation is performed when

[0085] It should be noted that data synchronization through data synchronization middleware ensures that data in different data sources remain consistent, that is, when data is updated, it can be synchronized in different data sources. Data synchronization middleware can also resolve data conflicts between different data sources and ensure data accuracy. During the data synchronization process, if an error occurs, the data synchronization middleware can record the error and recover to ensure data integrity and reliability.

[0086] It should be further explained that, in the specific implementation process, the central integration node reviews the received integration request files, performs data integration on the reviewed integration request files, and then generates multi-source comprehensive data, including:

[0087] The review performed by the central integration node on all received integration request files includes a first review operation and a second review operation;

[0088] The first audit operation includes document verification and content review;

[0089] The second audit operation includes permission checking and conflict detection;

[0090] Among them, file verification is used to verify whether the format, structure and integrity of the integration request file meet the predetermined standards. The role of content review is to review whether the data mapping, conversion logic and business rules in the integration request file are correct; permission check is used to confirm whether the distributed node initiating the request has the authority to perform data integration operations; conflict detection is used to review whether the audit request corresponding to the current integration request file conflicts with the existing data integration;

[0091] Execute different review decisions according to the passing status of the first review operation and the second review operation, wherein the review decisions include approval decision, rejection decision and feedback decision;

[0092] When both the first review operation and the second review operation are successful, the approval decision is executed, indicating that the review of the corresponding integration request file is passed and the data integration is approved;

[0093] When at least one of the first review operation and the second review operation fails, a rejection decision is executed, indicating that the review of the corresponding integration request file has not passed, and data integration is rejected, and rejection information is given, which is the specific data content that has not passed the review;

[0094] After the first review operation and the second review operation are completed, a feedback decision is executed. The content of the feedback decision is: the review result is fed back to each distributed node, and for the distributed node that executes the rejection decision, a modification suggestion for the corresponding integration request file is provided;

[0095] Data integration is performed on all approved integration request files at the central integration node to generate multi-source comprehensive data.

[0096] It should be further explained that, in the specific implementation process, the process of screening out abnormal integrated data points in the multi-source integrated data and correcting the multi-source integrated data at the abnormal integrated data points includes:

[0097] The multi-source integrated data is composed of data segments of a plurality of integrated data points, and the data segments of the plurality of integrated data points are numbered and the number is recorded as j, then j=1, 2, 3, ..., m, where m is a natural number greater than 0;

[0098] Starting from j=1, traverse the data segments corresponding to several integrated data points in order of number j from small to large, mark the integrated data points corresponding to the data segments with abnormalities as abnormal integrated data points, and correct the data of the data segments corresponding to the abnormal integrated data points;

[0099] The anomalies existing in the data segment include data missing, data duplication and data error. When the data segment at the corresponding abnormal integrated data point is corrected, the data missing, data duplication and data error existing in the data segment are resolved.

[0100] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for integrating heterogeneous information data, characterized in that: The following steps are involved: Step S1: configure the central integration node; Step S2: Create a distributed node for each of the multiple data sources, and all the distributed nodes obtain the node address of the central integration node and establish a connection relationship; Step S3: creating a corresponding node deployment environment for the distributed nodes that establish a connection relationship, and importing the heterogeneous information of the data source corresponding to each distributed node into the node deployment environment, and then generating respective integration request files; Step S4: Upload the integration request files of all distributed nodes to the central integration node; Step S5: The central integration node reviews the received integration request files, performs data integration on the reviewed integration request files, and generates multi-source integrated data; Step S6: Screen out abnormal integrated data points in the multi-source integrated data, and perform data correction on the multi-source integrated data at the abnormal integrated data points; The process of creating a corresponding node deployment environment for distributed nodes that establish a connection relationship includes: Each distributed node that has established a connection relationship is used as a deployment object for creating a node deployment environment. The operation process of creating a node deployment environment corresponding to the deployment object includes node environment construction, node resource configuration and middleware deployment, wherein the node environment construction, node resource configuration and middleware deployment are performed sequentially; The content of node environment construction is: build a node environment for each distributed node, initialize the node environment, convert the node environment to a data security state, and set the environment variables corresponding to the node environment; The content of node resource configuration is: allocate required node resources to the node environment that has completed environment initialization and setting of environment variables. The node resources include server, running storage space and network bandwidth; The content of middleware deployment is: deploying different types of data operation middleware related to data integration for each node environment corresponding to each distributed node, and the different types of data operation middleware include data synchronization middleware, data analysis middleware and data conversion middleware; The process of importing the heterogeneous information of the data source corresponding to each distributed node into the node deployment environment and then generating their respective integration request files includes: Import the heterogeneous information represented by the data source corresponding to each distributed node into the respective node deployment environment. When the heterogeneous information is detected in the node deployment environment, start all data operation middleware deployed in the corresponding node deployment environment. The data format of the heterogeneous information in the current node deployment environment is analyzed through the data analysis middleware. If the data format meets the preset final data format required for data integration, no operation is performed. Otherwise, a data conversion identifier is marked for the current heterogeneous information. The data conversion middleware receives all heterogeneous information processed by the data analysis middleware, and performs format conversion on the heterogeneous information marked with data conversion identifiers, thereby generating an initial integration request file for each distributed node, and binding the data synchronization middleware with the initial integration request file to generate a final integration request file; The process in which the central integration node reviews the received integration request files, performs data integration on the reviewed integration request files, and then generates multi-source comprehensive data includes: The central integration node reviews all received integration request files, including a first review operation and a second review operation, wherein the first review operation includes file verification and content review, and the second review operation includes authority check and conflict detection; According to the passing status of the first review operation and the second review operation, different review decisions are executed, and the review decisions include approval decision, rejection decision and feedback decision; When both the first review operation and the second review operation are successful, the approval decision is executed, indicating that the review of the corresponding integration request file is passed and the data integration is approved; When at least one of the first review operation and the second review operation fails, a rejection decision is executed, indicating that the review of the corresponding integration request file has not passed, and data integration is rejected, and rejection information is given, which is the specific data content that has not passed the review; After the first review operation and the second review operation are completed, a feedback decision is executed. The content of the feedback decision is: the review result is fed back to each distributed node, and for the distributed node that executes the rejection decision, a modification suggestion for the corresponding integration request file is provided; Data integration is performed on all approved integration request files at the central integration node to generate multi-source comprehensive data.

2. A heterogeneous information data integration method according to claim 1, characterized in that: The process of configuring a central integration node includes: Create a central integration node, set up the database of the central integration node and configure it. The configuration includes allocating database memory, configuring the database corresponding firewall information, and configuring the database permission access list; Create a node address for the central integration node. The node address is a dynamic IP address. The dynamic IP address is associated with a corresponding address list. The address list is used to record the address sequence corresponding to the dynamic IP address in different time periods. Configure the communication protocol for the central integration node, and perform node performance test on the central integration node after the communication protocol configuration is completed. The node performance test includes stress test, security test and disaster recovery test. When all node performance tests of the central integration node pass, the central integration node is configured successfully. Otherwise, continue to configure the central integration node until the central integration node is configured successfully.

3. A heterogeneous information data integration method according to claim 2, characterized in that: A distributed node is created for each of several data sources. The process of all distributed nodes obtaining the node address of the central integration node and establishing a connection relationship includes: Number several data sources and record them as i, i = 1, 2, 3, ..., n, where n is a natural number greater than 0, create a distributed node for each data source, and bind the source address of the data source to the distributed node, and record the source address bound to the distributed node created by the data source numbered i as Address[i]; Each distributed node obtains the node address of the central integration node according to the current time period, and establishes its own address mapping sequence pair with the central integration node. The address mapping sequence pair of the distributed node corresponding to the data source numbered i is recorded as G[i], G[i] =<Address[i],Address[Time]> , Address[Time] is the node address corresponding to the central integration node in the corresponding time period; When the address mapping sequence pair between the distributed node and the central integration node is established, the distributed node sends a connection establishment request file created by itself to the central integration node. After the central integration node receives the connection establishment request file, it enters its own communication protocol into the connection establishment request file and returns it to the distributed node that sent it. After the distributed node configures the returned communication protocol, it establishes a connection relationship between the distributed node and the central integration node.

4. A heterogeneous information data integration method according to claim 3, characterized in that: The process of uploading the integration request files of all distributed nodes to the central integration node includes: Upload the corresponding integration request file of each distributed node to the central integration node, and the data synchronization middleware bound to each integration request file performs real-time data synchronization. Data synchronization is performed according to the preset synchronization frequency, which is changed by the data maintenance personnel according to the actual data synchronization situation; Monitor the upload rate of each distributed node uploading the integration request file to the central integration node, and record the upload rate as V 上传 , set the alarm interval corresponding to the upload rate, and record the alarm interval as Ω 告警 , when V 上传 Ω 告警 When the rate abnormality alarm is generated, it will be pushed to the data maintenance personnel, who will adjust the upload rate to make V 上传 Ω 告警 , when V 上传 Ω 告警 No operation is performed when 5. A heterogeneous information data integration method according to claim 4, characterized in that: The process of screening out abnormal integrated data points in multi-source integrated data and correcting the multi-source integrated data at the abnormal integrated data points includes: The multi-source integrated data is composed of data segments of a plurality of integrated data points, and the data segments of the plurality of integrated data points are numbered and the number is recorded as j, then j=1, 2, 3, ..., m, where m is a natural number greater than 0; Starting from j=1, traverse the data segments corresponding to several integrated data points in order from small to large, mark the integrated data points corresponding to the data segments with abnormalities as abnormal integrated data points, and correct the data segments corresponding to the abnormal integrated data points. The anomalies in the data segments include data missing, data duplication, and data errors.

Images