Method for converging and sharing multi-source spatio-temporal data

Abstract

A method for gathering and sharing multi-source spatio-temporal data relates to the technical field of multi-source data fusion, and is characterized by comprising the following steps: data governance; performing data cataloguing; data storage; and issuing data. The method provided by the embodiment of the invention has the advantages that the structured non-spatial data of different industries or sources are accessed, and the name class and address class information items in the data are integrated with the place name address base provided by the invention according to the processes of data management, data coding, data storage and data release provided by the invention, so that the data processing efficiency is improved. And non-spatial data is converted into a data type with a spatial position. In the data management process, in order to guarantee that a mode which is as automatic as possible is adopted in the data management process, irregular data in names and addresses are preferentially eliminated through an automatic coding technology; and successfully coded data are automatically matched, and unsuccessfully coded data can be subjected to automatic falling graph modeling again through semantic matching.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source data fusion, specifically, by accessing structured non-spatial data from different industries or sources, and integrating the name and address information items in the data with the place name and address database provided by the present invention, according to the data governance, data coding, data storage and data publishing process provided by the present invention, the non-spatial data is transformed into a data type with a spatial location. In the data governance process, in order to ensure that the data governance process is automated as much as possible, the irregular data in the name and address are preferentially excluded through automatic coding technology; the successfully coded data is automatically matched, and the unsuccessful data can be automatically mapped and modeled again through semantic matching, etc., a method for the aggregation and sharing of multi-source spatiotemporal data. Background Art

[0006] At present, there are still several key technical problems in the multi-source data fusion technology. For example, how to effectively access structured non-spatial data from different industries or sources, integrate the name and address information items in the data with the provided place name and address database according to the provided data governance, data coding, data storage and data publishing process, and realize the transformation of non-spatial data into data types with spatial location. In the data governance process, in order to ensure that the data governance process is as automated as possible, the automatic coding technology is preferred to exclude irregular data in names and addresses; the successfully coded data is automatically matched, and the unsuccessful data can be automatically mapped and modeled again through semantic matching. In order to improve the continuity and maintainability of data and services, after adding the data cataloging and metadata management links, the basic information reading of the data storage and data publishing steps is completely based on the coding and metadata management links. The data storage and map container engine is built based on the C++ language. The data storage and map container docking part injects data increment monitoring and service registration discovery modules to completely solve the problem that the traditional spatial database and map container cannot be automatically docked and integrated. In the final service use step of the service, the security key and data stream encryption algorithm for data interface access are added to shield the risk of data leakage, which still needs to be solved. Summary of the invention

[0007] In order to overcome the drawbacks of the existing technical solutions, the structured non-spatial data of different industries or sources is accessed, and the name and address information items in the data are integrated with the place name and address library provided by the present invention according to the data governance, data encoding, data storage and data publishing process provided by the present invention to realize the conversion of non-spatial data into data types with spatial locations. In the data governance process, in order to ensure that the data governance process is as automated as possible, the irregular data in the name and address are preferentially excluded by automatic encoding technology; the successfully encoded data are automatically matched, and the unsuccessful data can be automatically mapped and modeled again through semantic matching. In order to improve the continuity and maintainability of data and services, the basic information reading of the data storage and data publishing steps is completely based on the encoding and metadata management links after adding the data cataloging and metadata management links. The data storage and map container engine is constructed based on the C++ language. The data storage and map container docking part injects the data increment monitoring and service registration discovery modules to completely solve the problem that the traditional spatial database and the map container cannot be automatically docked and integrated. In the final service use step of the service, the security key and data stream encryption algorithm for data interface access are added to shield the risk of data leakage.

[0008] A method for aggregating and sharing multi-source spatiotemporal data, the method comprising the following steps:

[0009] Data governance: extract place names or address fields from non-spatial data for standardization and name and address coding, calculate the matching values ​​of corresponding data entries by combining coding matching and semantic matching, and determine the database that the data entry needs to be included in according to the range of matching values;

[0010] Data cataloging: During the cache data management phase, the data results in the matched library are configured and extracted in turn, and the data catalog and metadata are mounted to form the preliminary preparation for data storage;

[0011] Data storage: Automatically read and interpret data levels to create data codes, and perform data quality inspection on the encoded data. Based on the conclusion of the quality inspection report, if the quality inspection conclusion is passed, the data synchronization storage process is executed; otherwise, the data enters the data update process, and the data that does not meet the quality inspection requirements executes the update method. After the data update is completed and the quality inspection is passed again, it enters the data storage queue;

[0012] Data publishing: Add monitoring publishing and map discovery anchors to the map container, automatically discover, publish, and update data services in an incremental manner, link the service list to service users in a service sharing manner, and use the AES encryption algorithm to encrypt and decrypt the data flow during service use.

[0013] A method for aggregating and sharing multi-source spatiotemporal data, wherein the specific steps of data governance are:

[0014] Name and address standardization: standardize the place name field or address field in the non-spatial data, parse and output the place name and address in a standardized manner according to the administrative division hierarchy, landmark names, and points of interest names preset in the database, so as to form an early reserve for data governance; the specific steps are to parse the place name and irregular address into a standardized address containing the province, city, district, county, street, road name, and house number; determine that the parsing result is a successful match, establish a matching field from province to house number in the database and assign a value; if the parsing result is a failed match, mark the data and process it through manual intervention tools, and finally mark the data that still cannot be parsed successfully as entering the semantic matching library and process it through semantic matching;

[0015] Name and address encoding: For data entries that have completed name and address resolution, encode each data entry in the form of province + city + district + county + street + road name + house number + serial number; for data entries that failed to be successfully encoded, mark them as being encoded through semantic matching;

[0016] The encoding operation formula is:

[0017] Code site =Code province +Code city +Code county +Code road +Code plate +Code order

[0018] Where: Code site A unique code representing a data entry;

[0019] Code province 、Code city 、Code county 、Code road 、Code plate Unique code values ​​representing province, city, district, county, street, road and house number respectively;

[0020] Code order Represents the data sequence number in the current data set;

[0021] Coding matching: Import the coded non-spatial data into the place name and address model matching library, and use the coding results of the name and address to segment and compare with the basic coding in the place name model library and the address model library. If the match is successful, it is marked as a correct match; if the match is not successful, it is marked as a match failure and the semantic matching process can be entered;

[0022] Semantic matching: For data that cannot be matched through coding matching, semantic segmentation matching is used. Semantic matching includes three processes: semantic segmentation, establishment of matching similarity matrix, and calculation of similarity matching results. The best matching place name and address model data is extracted based on the matching similarity value. When the matching similarity value is determined to be greater than 0.75, the default matching value result is valid, and the data is automatically extracted to the matched library; when the matching result is determined to be between 0.75 and 0.5, the data enters the manual intervention stage, and the validity of the matching result is confirmed through manual analysis; when the matching similarity value is determined to be lower than 0.5, the data entry is included in the unmatched library.

[0023] A method for aggregating and sharing multi-source spatiotemporal data, wherein the specific steps of data cataloging are:

[0024] Data directory: caches the data in the matched library in the data governance stage, extracts key items such as the source department, data industry classification, data format, and data timeliness of the matching data, and establishes the directory hierarchy of the matching data through automatic coding;

[0025] Metadata configuration and extraction: Metadata is extracted from the data that has been matched by data governance. Metadata is divided into management metadata, technical metadata, and business metadata. Management metadata is collected through aggregation tools, and the collected information items mainly include data source departments, data industry classifications, data permissions, and data time. Technical metadata and business metadata are mainly extracted through automatic extraction; technical metadata mainly extracts the number, size, and table name of data entries; business metadata mainly extracts the specific field name, type, precision, and default value of data entries;

[0026] Data mounting: Mount the newly created data directory and the corresponding metadata configuration information; use the data directory code as the only key to mount management metadata, technical metadata, and business metadata respectively to form preliminary preparations for data storage.

[0027] A method for aggregating and sharing multi-source spatiotemporal data, wherein the specific steps of storing the data are as follows:

[0028] Data encoding: Select the data directory node that the data governance results need to enter in the data directory, automatically read and interpret the data hierarchy, calculate the data encoding prefix, read the time characteristics of the data item, calculate the time component in the data encoding, and complete the automatic encoding process of geographic data;

[0029] Data quality inspection: By extracting the quality control rules required in the process of converting non-spatial data to spatial data, and based on the data directory and its attached metadata information, the governed data is input into the data quality inspection model, the quality inspection rules are automatically scheduled, and the quality inspection results are output; if the quality inspection result is judged to be passed, the data release step is entered; if the quality inspection result is judged to be failed, the data update step is entered, and the data processing environment that does not comply with the rules is carried out. After the quality inspection model is input again and passed, the data release step is entered;

[0030] Data update: Data that has not passed the quality inspection step is transferred to the data update step. If it is determined that there is data that has passed the quality inspection in the data of this step, this part of the data is not allowed to be updated online; otherwise, the data can be updated online;

[0031] Data synchronization: Data that passes the quality inspection step is transferred from the transit library to the data results library; the data is preloaded into the data synchronization queue in a cached manner and scheduled in batches by the synchronization task bus. If the data is incrementally synchronized, you can enter whether the backup parameters are required.

[0032] A method for aggregating and sharing multi-source spatiotemporal data, wherein the specific steps of data encoding are:

[0033] Automatic reading and interpretation of directory levels: Considering the complexity of spatiotemporal data classification, the data directory hierarchy is set to an unlimited level. After selecting the specific data directory corresponding to the governance outcome data, the directory hierarchy is automatically interpreted into a unique code that is progressively associated with each level through the interpretation algorithm, which serves as the unique identification code for the storage of governance data in the database;

[0034] The formula for automatic interpretation of directory levels is:

[0035] Decode(n)=Decode(n-1)+Decode(n-2)

[0036] Where:

[0037] Decode(n) is the data storage code, which is obtained by decoding according to the directory level;

[0038] n represents the hierarchical depth value of the directory corresponding to the current data storage, and n-1 represents the parent node of the n-1th level above the directory corresponding to the current data storage;

[0039] Reading of data item time characteristics: During the data aggregation process, the data time information in the management metadata is read, and the time identifier is converted into data version control information. The accuracy of the version number is dynamically adjusted according to the actual data update frequency. The accuracy can be controlled to the year, quarter, month, or day.

[0040] The formula for converting the time stamp to data version information is:

[0041] Time decode =Intercept(time,arg)

[0042] Where:

[0043] Time decode Represents the dynamic version number after the data time feature is extracted;

[0044] time represents the data time information in metadata;

[0045] arg represents the precision parameter of the current data, and the parameter values ​​are y: year, q: quarter, m: month, d: day;

[0046] Automatic coding of geographic data: The unique code in the directory level interpretation result is combined with the time identifier in the time characteristic of the data item. Before completing the data quality inspection, the code is interpreted as a temporary code, and the currently coded data is stored in the transit library; when the data named by the current code passes the data quality inspection link, the data is automatically synchronized to the results library;

[0047] The data encoding rule formula is:

[0048] T code =Catalog decode +Time decode

[0049] Where:

[0050] Unicode address Represents the unique code of geographic data;

[0051] Catalog decode Represents the current level code of the directory;

[0052] Time decode Represents the dynamic version number after the data time feature is extracted.

[0053] A method for aggregating and sharing multi-source spatiotemporal data, wherein the specific steps of data quality inspection are:

[0054] Quality inspection rule configuration: After configuring the quality control calculation rules required for the non-spatial data conversion process, the calculation of the entire quality inspection control process is modularized and subdivided through calculation logic such as grouping, splitting, and restriction to form a set of configurable quality control models; the quality inspection rules mainly include data integrity check, key feature check, and topological relationship check;

[0055] Quality inspection rule scheduling: By reading the business metadata and technical metadata attached to the target data directory of the data to be stored, they are compared item by item with the data to be inspected in the intermediate library, and the results of non-conforming items are analyzed and logged; if the overall analysis result is determined to be passed, the data synchronization step is entered; otherwise, the analysis result and log are output as a quality inspection report, and the data can enter the data update step.

[0056] A method for aggregating and sharing multi-source spatiotemporal data, wherein the specific steps of data publishing are:

[0057] Monitoring and publishing: embed database real-time monitoring subscription anchor points in the database; when an incremental data event occurs in the database, notify the map container of the data change event in the form of message communication; when the event type is data item increment, the map container triggers the service publishing mechanism and registers a new service in the map container; when the event type is data item increment, the map container triggers the data metadata update mechanism and subscribes and updates the registered services in the map container;

[0058] Map discovery: Inject the map discovery method into the map container. When an incremental subscription message appears in the service in the map container, the map service information in the container and the map metadata information are pushed to the map service sharing provider in the form of data packets through the map discovery method. The service provider provides multiple distributed service proxy clusters, which are distributedly deployed through the service gateway. The service provider determines the load and disconnects the access traffic in a timely manner.

[0059] Service sharing: The map service of the map discovery step pushes the incrementally changed map service to the map sharing step in the form of a service list; the map sharing step includes processes and methods such as service subscription, service query, service application and service approval; service subscription automatically pushes the service user's attention services based on the service user's subscription rules; service query matches the service list based on the query parameters passed in by the service user; service application provides a service application interface for service users; service approval provides a service approval interface for service managers;

[0060] Service usage: Service users use the services they have permission to use in their business according to the calling rules. During the service calling process, the security key provided by the service usage module is required, and the service is encrypted and decrypted according to the AES encryption algorithm during the service flow.

[0061] It can be seen from this that:

[0062] The method in the embodiment of the present invention is integrated with the name and address information items in the data and the place name and address library provided by the present invention according to the data governance, data coding, data storage and data publishing process provided by the present invention, so as to realize the conversion of non-spatial data into data types with spatial locations. In the data governance process, in order to ensure that the data governance process adopts the most automated method possible, the irregular data in the name and address are preferentially excluded by automatic coding technology; the successfully coded data are automatically matched, and the unsuccessful data can be automatically mapped and modeled again through semantic matching. In order to improve the continuity and maintainability of data and services, the basic information reading of the data storage and data publishing steps is completely based on the coding and metadata management links after adding the data cataloging and metadata management links. The data storage and map container engine is constructed based on the C++ language. The data storage and map container docking part injects the data increment monitoring and service registration discovery modules to completely solve the problem that the traditional spatial database and the map container cannot be automatically docked and integrated. In the final service use step of the service, the security key and data stream encryption algorithm for data interface access are added to shield the risk of data leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of a flow chart of a method for aggregating and sharing multi-source spatiotemporal data provided in an embodiment of the present invention;

[0064] Figure 2 A schematic diagram of a data management process in an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of a data cataloging process in an embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of the data storage process in an embodiment of the present invention;

[0067] Figure 5 A schematic diagram of a data encoding process in an embodiment of the present invention;

[0068] Figure 6 Schematic diagram of the data quality inspection process in an embodiment of the present invention;

[0069] Figure 7 The figure is a flowchart of the data publishing steps in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The exemplary embodiments and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0071] Embodiment 1:

[0072] Figure 1 A flow chart of the method for multi-source spatiotemporal data aggregation and sharing provided in this embodiment is as follows: Figure 1 As shown, a method for aggregating and sharing multi-source spatiotemporal data includes the following steps:

[0073] Data governance: extract place names or address fields from non-spatial data for standardization and name and address coding, calculate the matching values ​​of corresponding data entries by combining coding matching and semantic matching, and determine the database that the data entry needs to be included in according to the range of matching values;

[0074] Data cataloging: During the cache data management phase, the data results in the matched library are configured and extracted in turn, and the data catalog and metadata are mounted to form the preliminary preparation for data storage;

[0075] Data storage: Automatically read and interpret data levels to create data codes, and perform data quality inspection on the encoded data. Based on the conclusion of the quality inspection report, if the quality inspection conclusion is passed, the data synchronization storage process is executed; otherwise, the data enters the data update process, and the data that does not meet the quality inspection requirements executes the update method. After the data update is completed and the quality inspection is passed again, it enters the data storage queue;

[0076] Data publishing: Add monitoring publishing and map discovery anchors to the map container, automatically discover, publish, and update data services in an incremental manner, link the service list to service users in a service sharing manner, and use the AES encryption algorithm to encrypt and decrypt the data flow during service use.

[0077] like Figure 2 As shown, a method for aggregating and sharing multi-source spatiotemporal data is provided, and the specific steps of data governance are as follows:

[0078] Name and address standardization: standardize the place name field or address field in the non-spatial data, parse and output the place name and address in a standardized manner according to the administrative division hierarchy, landmark names, and points of interest names preset in the database, so as to form an early reserve for data governance; the specific steps are to parse the place name and irregular address into a standardized address containing the province, city, district, county, street, road name, and house number; determine that the parsing result is a successful match, establish a matching field from province to house number in the database and assign a value; if the parsing result is a failed match, mark the data and process it through manual intervention tools, and finally mark the data that still cannot be parsed successfully as entering the semantic matching library and process it through semantic matching;

[0079] Name and address encoding: For data entries that have completed name and address resolution, encode each data entry in the form of province + city + district + county + street + road name + house number + serial number; for data entries that failed to be successfully encoded, mark them as being encoded through semantic matching;

[0080] The encoding operation formula is:

[0081] Code site =Code province +Code city +Code county +Code road +Code plate +Code order

[0082] Where: Code site A unique code representing a data entry;

[0083] Code province 、Code city 、Code county 、Code road 、Code plate Unique code values ​​representing province, city, district, county, street, road and house number respectively;

[0084] Code order Represents the data sequence number in the current data set;

[0085] Coding matching: Import the coded non-spatial data into the place name and address model matching library, and use the coding results of the name and address to segment and compare with the basic coding in the place name model library and the address model library. If the match is successful, it is marked as a correct match; if the match is not successful, it is marked as a match failure and the semantic matching process can be entered;

[0086] Semantic matching: For data that cannot be matched through coding matching, semantic segmentation matching is used. Semantic matching includes three processes: semantic segmentation, establishment of matching similarity matrix, and calculation of similarity matching results. The best matching place name and address model data is extracted based on the matching similarity value. When the matching similarity value is determined to be greater than 0.75, the default matching value result is valid, and the data is automatically extracted to the matched library; when the matching result is determined to be between 0.75 and 0.5, the data enters the manual intervention stage, and the validity of the matching result is confirmed through manual analysis; when the matching similarity value is determined to be lower than 0.5, the data entry is included in the unmatched library.

[0087] like Figure 3 As shown, a method for aggregating and sharing multi-source spatiotemporal data, the specific steps of data cataloging are:

[0088] Data directory: caches the data in the matched library in the data governance stage, extracts key items such as the source department, data industry classification, data format, and data timeliness of the matching data, and establishes the directory hierarchy of the matching data through automatic coding;

[0089] Metadata configuration and extraction: Metadata is extracted from the data that has been matched by data governance. Metadata is divided into management metadata, technical metadata, and business metadata. Management metadata is collected through aggregation tools, and the collected information items mainly include data source departments, data industry classifications, data permissions, and data time. Technical metadata and business metadata are mainly extracted through automatic extraction; technical metadata mainly extracts the number, size, and table name of data entries; business metadata mainly extracts the specific field name, type, precision, and default value of data entries;

[0090] Data mounting: Mount the newly created data directory and the corresponding metadata configuration information; use the data directory code as the only key to mount management metadata, technical metadata, and business metadata respectively to form preliminary preparations for data storage.

[0091] like Figure 4 As shown, a method for aggregating and sharing multi-source spatiotemporal data, the specific steps of data storage are:

[0092] Data encoding: Select the data directory node that the data governance results need to enter in the data directory, automatically read and interpret the data hierarchy, calculate the data encoding prefix, read the time characteristics of the data item, calculate the time component in the data encoding, and complete the automatic encoding process of geographic data;

[0093] Data quality inspection: By extracting the quality control rules required in the process of converting non-spatial data to spatial data, and based on the data directory and its attached metadata information, the governed data is input into the data quality inspection model, the quality inspection rules are automatically scheduled, and the quality inspection results are output; if the quality inspection result is judged to be passed, the data release step is entered; if the quality inspection result is judged to be failed, the data update step is entered, and the data processing environment that does not comply with the rules is carried out. After the quality inspection model is input again and passed, the data release step is entered;

[0094] Data update: Data that has not passed the quality inspection step is transferred to the data update step. If it is determined that there is data that has passed the quality inspection in the data of this step, this part of the data is not allowed to be updated online; otherwise, the data can be updated online;

[0095] Data synchronization: Data that passes the quality inspection step is transferred from the transit library to the data results library; the data is preloaded into the data synchronization queue in a cached manner and scheduled in batches by the synchronization task bus. If the data is incrementally synchronized, you can enter whether the backup parameters are required.

[0096] like Figure 5 As shown, a method for aggregating and sharing multi-source spatiotemporal data, the specific steps of data encoding are:

[0097] Automatic reading and interpretation of directory levels: Considering the complexity of spatiotemporal data classification, the data directory hierarchy is set to an unlimited level. After selecting the specific data directory corresponding to the governance outcome data, the directory hierarchy is automatically interpreted into a unique code that is progressively associated with each level through the interpretation algorithm, which serves as the unique identification code for the storage of governance data in the database;

[0098] The formula for automatic interpretation of directory levels is:

[0099] Decode(n)=Decode(n-1)+Decode(n-2)

[0100] Where:

[0101] Decode(n) is the data storage code, which is obtained by decoding according to the directory level;

[0102] n represents the hierarchical depth value of the directory corresponding to the current data storage, and n-1 represents the parent node of the n-1th level above the directory corresponding to the current data storage;

[0103] Reading of data item time characteristics: During the data aggregation process, the data time information in the management metadata is read, and the time identifier is converted into data version control information. The accuracy of the version number is dynamically adjusted according to the actual data update frequency. The accuracy can be controlled to the year, quarter, month, or day.

[0104] The formula for converting the time stamp to data version information is:

[0105] Time decode =Intercept(time,arg)

[0106] Where:

[0107] Time decode Represents the dynamic version number after the data time feature is extracted;

[0108] time represents the data time information in metadata;

[0109] arg represents the precision parameter of the current data, and the parameter values ​​are y: year, q: quarter, m: month, d: day;

[0110] Automatic coding of geographic data: The unique code in the directory level interpretation result is combined with the time identifier in the time characteristic of the data item. Before completing the data quality inspection, the code is interpreted as a temporary code, and the currently coded data is stored in the transit library; when the data named by the current code passes the data quality inspection link, the data is automatically synchronized to the results library;

[0111] The data encoding rule formula is:

[0112] T code =Catalog decode +Time decode

[0113] Where:

[0114] Unicode address Represents the unique code of geographic data;

[0115] Catalog decode Represents the current level code of the directory;

[0116] Time decode Represents the dynamic version number after the data time feature is extracted.

[0117] like Figure 6 As shown, a method for aggregating and sharing multi-source spatiotemporal data, the specific steps of data quality inspection are:

[0118] Quality inspection rule configuration: After configuring the quality control calculation rules required for the non-spatial data conversion process, the calculation of the entire quality inspection control process is modularized and subdivided through calculation logic such as grouping, splitting, and restriction to form a set of configurable quality control models; the quality inspection rules mainly include data integrity check, key feature check, and topological relationship check;

[0119] Quality inspection rule scheduling: By reading the business metadata and technical metadata attached to the target data directory of the data to be stored, they are compared item by item with the data to be inspected in the intermediate library, and the results of non-conforming items are analyzed and logged; if the overall analysis result is determined to be passed, the data synchronization step is entered; otherwise, the analysis result and log are output as a quality inspection report, and the data can enter the data update step.

[0120] like Figure 7 As shown, a method for aggregating and sharing multi-source spatiotemporal data, the specific steps of data publishing are:

[0121] Monitoring and publishing: embed database real-time monitoring subscription anchor points in the database; when an incremental data event occurs in the database, notify the map container of the data change event in the form of message communication; when the event type is data item increment, the map container triggers the service publishing mechanism and registers a new service in the map container; when the event type is data item increment, the map container triggers the data metadata update mechanism and subscribes and updates the registered services in the map container;

[0122] Map discovery: Inject the map discovery method into the map container. When an incremental subscription message appears in the service in the map container, the map service information in the container and the map metadata information are pushed to the map service sharing provider in the form of data packets through the map discovery method. The service provider provides multiple distributed service proxy clusters, which are distributedly deployed through the service gateway. The service provider determines the load and disconnects the access traffic in a timely manner.

[0123] Service sharing: The map service of the map discovery step pushes the incrementally changed map service to the map sharing step in the form of a service list; the map sharing step includes processes and methods such as service subscription, service query, service application and service approval; service subscription automatically pushes the service user's attention services based on the service user's subscription rules; service query matches the service list based on the query parameters passed in by the service user; service application provides a service application interface for service users; service approval provides a service approval interface for service managers;

[0124] Service usage: Service users use the services they have permission to use in their business according to the calling rules. During the service calling process, the security key provided by the service usage module is required, and the service is encrypted and decrypted according to the AES encryption algorithm during the service flow.

[0125] The following is an example of a specific implementation:

[0126] The foregoing method descriptions and flow charts are provided only as illustrative examples and are not intended to require or imply that the above operations or steps of various aspects must be performed in the order given. As will be appreciated by those skilled in the art, the order of the boxes in the foregoing aspects may be performed in any order. Words such as "thereafter," "then," "next," and the like are not intended to limit the order of operations or steps; these words are only used to guide the reader through the description of the method. In addition, any singular reference to a claim element, for example, the use of the article "a," "an," or "the" is not to be construed as limiting the element to the singular.

[0127] The various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the aspects disclosed herein can all be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly represent the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are generally described above around their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire method. A skilled person can implement the described functions in an alternative manner for each specific application, but such implementation decisions should not be interpreted as causing a departure from the scope of protection of the present invention.

[0128] The method in the embodiment of the present invention is integrated with the name and address information items in the data and the place name and address library provided by the present invention according to the data governance, data coding, data storage and data publishing process provided by the present invention, so as to realize the conversion of non-spatial data into data types with spatial locations. In the data governance process, in order to ensure that the data governance process adopts the most automated method possible, the irregular data in the name and address are preferentially excluded by automatic coding technology; the successfully coded data are automatically matched, and the unsuccessful data can be automatically mapped and modeled again through semantic matching. In order to improve the continuity and maintainability of data and services, the basic information reading of the data storage and data publishing steps is completely based on the coding and metadata management links after adding the data cataloging and metadata management links. The data storage and map container engine is constructed based on the C++ language. The data storage and map container docking part injects the data increment monitoring and service registration discovery modules to completely solve the problem that the traditional spatial database and the map container cannot be automatically docked and integrated. In the final service use step of the service, the security key and data stream encryption algorithm for data interface access are added to shield the risk of data leakage.

[0129] The foregoing description of the disclosed aspects is provided to enable any person skilled in the art to implement or use the present invention. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects provided herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for aggregating and sharing multi-source spatiotemporal data, characterized in that The method comprises the following steps: Data governance: extract place names or address fields from non-spatial data for standardization and name and address coding, calculate the matching values ​​of corresponding data entries by combining coding matching and semantic matching, and determine the database that the data entry needs to be included in according to the range of matching values; Data cataloging: During the cache data management phase, the data results in the matched library are configured and extracted in turn, and the data catalog and metadata are mounted to form the preliminary preparation for data storage; Data storage: Automatically read and interpret data levels to create data codes, and perform data quality inspection on the encoded data. Based on the conclusion of the quality inspection report, if the quality inspection conclusion is passed, the data synchronization storage process is executed; otherwise, the data enters the data update process, and the data that does not meet the quality inspection requirements executes the update method. After the data update is completed and the quality inspection is passed again, it enters the data storage queue; Data publishing: Add monitoring publishing and map discovery anchors to the map container, automatically discover, publish, and update data services in an incremental manner, link the service list to service users in a service sharing manner, and use the AES encryption algorithm to encrypt and decrypt the data flow during service use.

2. The method for multi-source spatiotemporal data aggregation and sharing according to claim 1, characterized in that: The specific steps of data governance are: Name and address standardization: standardize the place name field or address field in non-spatial data, parse and output the place name and address in a standardized manner according to the administrative division hierarchy, landmark names, and points of interest names preset in the database, so as to form an early reserve for data governance; the specific steps are to parse the place name and irregular address into a standardized address containing the province, city, district, county, street, road name, and house number; determine that the parsing result is a successful match, establish a matching field from province to house number in the database and assign a value; if the parsing result is a failed match, mark the data and process it through manual intervention tools, and finally mark the data that still cannot be parsed successfully as entering the semantic matching library and process it through semantic matching; Name and address encoding: For data entries that have completed name and address resolution, encode each data entry in the form of province + city + district + county + street + road name + house number + serial number; for data entries that failed to be successfully encoded, mark them as being encoded through semantic matching; The encoding operation formula is: Code site =Code province +Code city +Code county +Code road +Code plate +Code order Where: Code site A unique code representing a data entry; Code province 、Code city 、Code county 、Code road 、Code plate Unique code values ​​representing province, city, district, county, street, road and house number respectively; Code order Represents the data sequence number in the current data set; Coding matching: Import the coded non-spatial data into the place name and address model matching library, and use the coding results of the name and address to segment and compare with the basic coding in the place name model library and the address model library. If the match is successful, it is marked as a correct match; if the match is not successful, it is marked as a match failure and the semantic matching process can be entered; Semantic matching: For data that cannot be matched through coding matching, semantic segmentation matching is used. Semantic matching includes three processes: semantic segmentation, establishment of matching similarity matrix, and calculation of similarity matching results. The best matching place name and address model data is extracted based on the matching similarity value. When the matching similarity value is determined to be greater than 0.75, the default matching value result is valid, and the data is automatically extracted to the matched library; when the matching result is determined to be between 0.75 and 0.5, the data enters the manual intervention stage, and the validity of the matching result is confirmed through manual analysis; when the matching similarity value is determined to be lower than 0.5, the data entry is included in the unmatched library.

3. The method for multi-source spatiotemporal data aggregation and sharing according to claim 1, characterized in that: The specific steps of data cataloging are: Data directory: caches the data in the matched library in the data governance stage, extracts key items such as the source department, data industry classification, data format, and data timeliness of the matching data, and establishes the directory hierarchy of the matching data through automatic coding; Metadata configuration and extraction: Metadata is extracted from the data that has been matched by data governance. Metadata is divided into management metadata, technical metadata, and business metadata. Management metadata is collected through aggregation tools, and the collected information items mainly include data source departments, data industry classifications, data permissions, and data time. Technical metadata and business metadata are mainly extracted through automatic extraction; technical metadata mainly extracts the number, size, and table name of data entries; business metadata mainly extracts the specific field name, type, precision, and default value of data entries; Data mounting: Mount the newly created data directory and the corresponding metadata configuration information; use the data directory code as the only key to mount management metadata, technical metadata, and business metadata respectively, forming preliminary preparations for data storage.

4. The method for multi-source spatiotemporal data aggregation and sharing according to claim 1, characterized in that: The specific steps of data storage are: Data encoding: Select the data directory node that the data governance results need to enter in the data directory, automatically read and interpret the data hierarchy, calculate the data encoding prefix, read the time characteristics of the data item, calculate the time component in the data encoding, and complete the automatic encoding process of geographic data; Data quality inspection: By extracting the quality control rules required in the process of converting non-spatial data to spatial data, and based on the data directory and its attached metadata information, the governed data is input into the data quality inspection model, the quality inspection rules are automatically scheduled, and the quality inspection results are output; if the quality inspection result is judged to be passed, the data release step is entered; if the quality inspection result is judged to be failed, the data update step is entered, and the data processing environment that does not comply with the rules is carried out. After the quality inspection model is input again and passed, the data release step is entered; Data update: Data that has not passed the quality inspection step is transferred to the data update step. If it is determined that there is data that has passed the quality inspection in the data of this step, this part of the data is not allowed to be updated online; otherwise, the data can be updated online; Data synchronization: Data that passes the quality inspection step is transferred from the transit library to the data results library; the data is preloaded into the data synchronization queue in a cached manner and scheduled in batches by the synchronization task bus. If the data is incrementally synchronized, you can enter whether the backup parameters are required.

5. The method for multi-source spatiotemporal data aggregation and sharing according to claim 4, characterized in that: The specific steps of data encoding are: Automatic reading and interpretation of directory levels: Considering the complexity of spatiotemporal data classification, the data directory hierarchy is set to an unlimited level. After selecting the specific data directory corresponding to the governance outcome data, the directory hierarchy is automatically interpreted into a unique code that is progressively associated with each level through the interpretation algorithm, which serves as the unique identification code for the storage of governance data in the database; The formula for automatic interpretation of directory levels is: Decode(n)=Decode(n-1)+Decode(n-2) Where: Decode(n) is the data storage code, which is obtained by decoding according to the directory level; n represents the hierarchical depth value of the directory corresponding to the current data storage, and n-1 represents the parent node of the n-1th level above the directory corresponding to the current data storage; Reading of data item time characteristics: During the data aggregation process, the data time information in the management metadata is read, and the time identifier is converted into data version control information. The accuracy of the version number is dynamically adjusted according to the actual data update frequency. The accuracy can be controlled to the year, quarter, month, or day. The formula for converting the time stamp to data version information is: Time decode =Intercept(time,arg) Where: Time decode Represents the dynamic version number after the data time feature is extracted; time represents the data time information in metadata; arg represents the precision parameter of the current data, and the parameter values ​​are y: year, q: quarter, m: month, d: day; Automatic coding of geographic data: The unique code in the directory level interpretation result is combined with the time identifier in the time characteristic of the data item. Before completing the data quality inspection, the code is interpreted as a temporary code, and the currently coded data is stored in the transit library; when the data named by the current code passes the data quality inspection link, the data is automatically synchronized to the results library; The data encoding rule formula is: T code =Catalog decode +Time decode Where: Unicode address Represents the unique code of geographic data; Catalog decode Represents the current level code of the directory; Time decode Represents the dynamic version number after the data time feature is extracted.

6. The method for multi-source spatiotemporal data aggregation and sharing according to claim 4, characterized in that: The specific steps of the data quality inspection are: Quality inspection rule configuration: After configuring the quality control calculation rules required for the non-spatial data conversion process, the calculation of the entire quality inspection control process is modularized and subdivided through calculation logic such as grouping, splitting, and restriction to form a set of configurable quality control models; the quality inspection rules mainly include data integrity check, key feature check, and topological relationship check; Quality inspection rule scheduling: By reading the business metadata and technical metadata attached to the target data directory of the data to be stored, the data is compared item by item with the data to be inspected in the intermediate library, and the results of non-conforming items are analyzed and recorded in the log; the overall analysis result is determined to be passed, and the data synchronization step is entered; Otherwise, the analysis results and logs are output as a quality inspection report, and the data can enter the data update step.

7. The method for multi-source spatiotemporal data aggregation and sharing according to claim 1, characterized in that: The specific steps of data release are: Monitoring and publishing: embed database real-time monitoring subscription anchor points in the database; When an incremental data event occurs in the database, the data change event is notified to the map container in the form of message communication; when the event type is a data item increment, the map container triggers the service publishing mechanism and registers a new service in the map container; when the event type is a data item increment, the map container triggers the data metadata update mechanism and subscribes to and updates the registered services in the map container; Map discovery: Inject the map discovery method into the map container. When an incremental subscription message appears in the service in the map container, the map service information in the container and the map metadata information are pushed to the map service sharing provider in the form of data packets through the map discovery method. The service provider provides multiple distributed service proxy clusters, which are deployed in a distributed manner through service gateways. They can independently judge the load and perform timely fuses based on the access traffic. Service sharing: push the map service of the map discovery step and the map service with incremental changes to the map sharing step in the form of a service list; The map sharing steps include service subscription, service query, service application and service approval processes and methods; Service subscription automatically pushes the services that service users are interested in based on their subscription rules; service query matches the service list based on the query parameters passed in by the service user; service application provides a service application interface for service users; service approval provides a service approval interface for service managers; Service use: Service users use the services they have permission to use in their business according to the calling rules; During the service call process, the security key provided by the service module needs to be used, and the service encryption and decryption are performed according to the AES encryption algorithm during the service flow.

Images