Space-time alignment and semantic association modeling method for cross-platform geographic information data

Abstract

The invention relates to a space-time alignment and semantic association modeling method for cross-platform geographic information data, and belongs to the field of big data and natural language processing. The method comprises the following steps: step 1, carrying out space-time uncertainty modeling and probabilistic representation, and quantifying time and space uncertainty of multi-source data; 2, combining an alignment decision mechanism with adaptive learning to complete multi-evidence intelligent flexible alignment fusion; step 3, carrying out deep semantic analysis and real-time association discovery based on multi-modal semantic understanding and association mining; and 4, dynamically updating and reasoning the knowledge graph, and performing real-time analysis and real-time reasoning. According to the method, for a sudden search task, a new data source can be rapidly integrated, the knowledge graph is enriched, global situation visual control is achieved, multilevel reasoning based on the knowledge graph is achieved, efficient and accurate geographic information data intelligence is supported, and the task response efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the field of big data and natural language processing, specifically relating to a method for spatiotemporal alignment and semantic association modeling of cross-platform geographic information data. Background Technology

[0002] Currently, geographic information data processing primarily employs a layered processing architecture. For multi-source heterogeneous data such as satellite remote sensing data, UAV imagery, BeiDou or GPS positioning information, meteorological sensor data, and voice or text reports, after preprocessing operations including data parsing, noise removal, format unification and alignment, and coordinate unification and alignment, a rigid alignment algorithm based on precise spatiotemporal reference is used to convert the timestamps of all data into a unified standard time format and their spatial coordinates into the same geodetic coordinate system. At the semantic processing level, methods based on keyword matching and predefined classification rules are mainly used. A pre-defined keyword library is used to filter text and annotate key information, mapping it to static spatiotemporal coordinates. Then, an ETL data pipeline or rule engine is used to import the processing results into a GIS platform or command and decision-making system for visualization and query analysis. The entire process is rule-driven and requires data structuring and spatiotemporal standardization. The current solution has the following shortcomings:

[0003] (1) Rigid spatiotemporal alignment mechanism cannot effectively handle spatiotemporal uncertainty of geographic information data, including practical problems such as positioning equipment error, data transmission delay, and difference in sensing range, which leads to multi-source data with inherent correlation being unable to be correctly matched due to micro-spatiotemporal deviation;

[0004] (2) The semantic processing method based on keyword matching lacks the ability to understand the language context and deep semantics, and cannot effectively deal with the representational differences of synonyms such as "in danger" and "trapped", nor can it extract deep semantic information from unstructured data such as images and speech.

[0005] (3) The entire processing flow relies heavily on predefined rule systems and mapping models, lacks adaptive learning capabilities, and is difficult to cope with complex application environments with dynamic changes and diverse data source types. The system has insufficient scalability and flexibility.

[0006] (4) The failure to construct a unified data association model with reasoning ability resulted in fragmented information, which could not provide situational awareness support for task decision-making with in-depth and multi-dimensional evidence. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] The technical problem this invention aims to solve is how to provide a cross-platform method for spatiotemporal alignment and semantic association modeling of geographic information data. This method addresses the problems in existing geographic information data processing, such as the inability of rigid spatiotemporal alignment mechanisms to effectively handle the spatiotemporal uncertainty of geographic information data, the lack of understanding of language context and deep semantics in keyword matching-based semantic processing methods, the heavy reliance of the entire processing flow on predefined rule systems and mapping models, and the failure to construct a unified data association model with reasoning capabilities.

[0009] (II) Technical Solution

[0010] To address the aforementioned technical problems, this invention proposes a method for spatiotemporal alignment and semantic association modeling of cross-platform geographic information data, which includes the following steps:

[0011] Step 1: Spatiotemporal uncertainty modeling and probabilistic representation to quantify the temporal and spatial uncertainties of multi-source data: For satellite positioning data, construct elliptical confidence regions using HDOP values ​​and error parameters; parse geographic information into spatial ranges using natural language processing technology; establish a time window model based on data source characteristics, considering the characteristics of the time data sources; and establish a spatiotemporal probability distribution map of data points using differentiated spatiotemporal uncertainty modeling methods, characterizing spatiotemporal uncertainty through probability density functions.

[0012] Step 2: Combine the alignment decision mechanism with adaptive learning to complete the intelligent flexible alignment fusion of multiple evidences: Based on the calculation of the spatiotemporal correlation of different data objects, the vectors generated by the domain-optimized pre-trained model are used to calculate the cosine similarity to represent the semantic similarity of different data objects. The spatiotemporal correlation and semantic similarity are weighted and fused by a neural network fusion machine to output the comprehensive confidence of the relationship between different data objects.

[0013] Step 3: Based on multimodal semantic understanding and association mining, conduct deep semantic analysis and real-time association discovery: use a domain adaptive model to realize context-aware text representation of text data; use an image processing model to process image data, and simultaneously realize object detection and scene understanding to obtain image features; use a cross-attention mechanism to realize fine-grained semantic alignment of text and images, and use a pointer network to realize the identification of relationships between entities, and construct knowledge graph fact triples;

[0014] Step 4: Dynamic updating and reasoning of the knowledge graph, real-time analysis and reasoning: Through an incremental update mechanism, newly added data is processed by the alignment and association module to obtain candidate fact triples. After being screened by the quality assessment module, the data is updated into the knowledge graph. The graph data is processed by a time-aware graph neural network to realize real-time updating and reasoning of the graph, enabling continuous learning and adaptive evolution.

[0015] (III) Beneficial Effects

[0016] This invention proposes a spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data. Compared with previous methods, this invention effectively improves the fusion effect of geographic information data. Addressing the uncertainty of geographic information data, this invention establishes a refined probability model, utilizes a probability density function to achieve flexible alignment, and combines deep semantic modeling to achieve the optimal combination of spatiotemporal and semantic evidence, effectively reducing spatiotemporal bias and improving the accuracy and coverage of multi-source information association. This invention enables multimodal semantic understanding and association mining for geographic information data, based on time-aware knowledge graph dynamic updates and reasoning. It can quickly integrate new data sources to enrich the knowledge graph for tasks such as sudden location and search, achieving global situational awareness and enabling multi-level reasoning based on the knowledge graph, supporting efficient and accurate geographic information data intelligence, and improving task response efficiency. Attached Figure Description

[0017] Figure 1 This is the overall architecture of the present invention;

[0018] Figure 2 This is a flowchart illustrating the spatiotemporal uncertainty modeling and probabilistic representation of the present invention.

[0019] Figure 3 This is a flowchart illustrating the joint alignment decision-making mechanism and adaptive learning process of the present invention.

[0020] Figure 4 This is a flowchart of the multimodal semantic understanding and association mining process of the present invention;

[0021] Figure 5 This is a flowchart illustrating the dynamic updating and reasoning process of the knowledge graph in this invention. Detailed Implementation

[0022] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0023] This invention proposes a spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data. It constructs a flexible alignment mechanism that allows for spatiotemporal uncertainties, improving the association accuracy of multi-source data in real-world application scenarios. It achieves deep semantic understanding and association across multimodal data such as text, images, and sensor signals, and builds an intelligent fusion framework based on automated and adaptive updates. This reduces the application of manual rules, supports the establishment of a geographic information knowledge graph that supports comprehensive reasoning, and provides timely, effective, comprehensive, and reliable data support and intelligent analysis methods for task decision-making.

[0024] To overcome the aforementioned defects and shortcomings, this invention proposes a cross-platform method for spatiotemporal alignment and semantic association modeling of geographic information data. This is achieved by implementing a task-oriented hierarchical intelligent fusion framework, and the method includes the following steps:

[0025] Step 1: Spatiotemporal uncertainty modeling and probabilistic representation to quantify the temporal and spatial uncertainties of multi-source data: For satellite positioning data, construct elliptical confidence regions using HDOP values ​​and error parameters; parse geographic information into spatial ranges using natural language processing technology; establish a time window model based on data source characteristics, considering the characteristics of time data sources; and establish a spatiotemporal probability distribution map of data points using differentiated spatiotemporal uncertainty modeling methods, characterizing spatiotemporal uncertainty through probability density functions as the basis for subsequent flexible alignment.

[0026] Step 2: Joint alignment decision mechanism and adaptive learning to complete multi-evidence intelligent flexible alignment fusion: Based on the calculation of the spatiotemporal correlation of different data objects, the vectors generated by the domain-optimized pre-trained model are used to calculate the cosine similarity to represent the semantic similarity of different data objects. The spatiotemporal correlation and semantic similarity are weighted and fused by a neural network fusion machine to output the comprehensive confidence of the relationship between different data objects.

[0027] Step 3: Based on multimodal semantic understanding and association mining, conduct deep semantic analysis and real-time association discovery: use a domain adaptive model to realize context-aware text representation of text data; use an image processing model to process image data, and simultaneously realize object detection and scene understanding to obtain image features; use a cross-attention mechanism to realize fine-grained semantic alignment of text and images, and use a pointer network to realize the identification of relationships between entities, and construct knowledge graph fact triples.

[0028] Step 4: Dynamic Update and Reasoning of the Knowledge Graph, Real-time Analysis and Reasoning: Through an incremental update mechanism, newly added data is processed by the alignment and association module to obtain candidate fact triples. After being filtered by the quality assessment module, the data is updated into the knowledge graph. The graph data is processed by a time-aware graph neural network (T-GNN) to achieve real-time updates and reasoning of the graph, enabling continuous learning and adaptive evolution.

[0029] Example 1:

[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Step 1: Spatiotemporal uncertainty modeling and probabilistic representation based on probability distribution

[0032] To address the uncertainty of geographic information data, a refined probability model is established.

[0033] In terms of spatial uncertainty modeling, for satellite positioning data, a probabilistic model based on error propagation theory is used. Based on the horizontal accuracy factor HDOP provided by the positioning equipment used according to the geometric distribution of available satellites, an elliptical confidence region based on the probability distribution of positioning points is established.

[0034] The semi-axis lengths of the ellipse are respectively and .

[0035] in This is the confidence level coefficient, typically taken as 2.0 to 3.0. and The standard deviations of the measurement errors of the positioning device in the east-west and north-south directions are used to accurately describe the probability distribution of the positioning point within the elliptical confidence region through the covariance matrix output by the positioning device:

[0036]

[0037] in, Let be the covariance matrix, where for and The correlation coefficient.

[0038] For geographic information descriptions, natural language processing technology is used to parse out the location description part, and GIS is used to draw polygon confidence regions. The radius of the polygon is determined by the strength of the descriptive words, such as "nearby" corresponding to 200 meters, "a belt" corresponding to 500 meters, etc.

[0039] To model time uncertainty, a time window model based on the characteristics of the time data source is established. Specifically, sensor data is assumed to follow a normal distribution. Where μ is the timestamp and σ is half of the acquisition period. Manually reported data uses a uniform distribution. ,in, For the time of data collection or recording, Determined based on report type and experience value.

[0040] By modeling temporal and spatial uncertainties, a spatiotemporal probability distribution map is generated for each location point. The spatiotemporal uncertainty is characterized by a probability density function and used for subsequent alignment analysis.

[0041] Step 2: Joint Alignment Decision Mechanism Based on Multi-Evidence Fusion and Adaptive Learning

[0042] This invention designs a joint alignment decision mechanism based on multi-evidence fusion, which achieves the optimal combination of spatiotemporal and semantic evidence through a neural network fusioner.

[0043] The spatiotemporal correlation degree is calculated using an improved probability density function overlap integral method:

[0044]

[0045] in and The positions are respectively The time is The spatiotemporal probability density functions of the two data objects and Its spatiotemporal domain.

[0046] Semantic similarity calculation employs a Siamese network structure based on AdaptBERT. Building upon AdaptBERT, it utilizes a large-scale geographic information text corpus for domain-adaptive training to generate geographic information-specific semantic vectors. Cosine similarity is then used to calculate the semantic association strength described by different location data. .

[0047] Multi-evidence fusion is achieved by constructing a three-layer fully connected neural network consisting of an input layer, a hidden layer, and an output layer, using a binary cross-entropy loss function.

[0048]

[0049] in, The network parameters are obtained by training with historical geographic information data, and the input features are spatiotemporal correlation and semantic similarity. Finally, the overall confidence score within the range of (0, 1) is output, representing the strength of the data association. This association strength is then compared with the decision threshold. Compare the two data points to determine if they are related.

[0050] This invention also introduces a dynamic learning mechanism, which continuously learns online through real-time feedback information of the alignment results and dynamically adjusts the decision threshold. .

[0051]

[0052] in For the current accuracy rate, For the target accuracy, This is the learning rate. This method allows for adaptation to different task environments.

[0053] Step 3: Multimodal Semantic Understanding and Association Mining for Geographic Information Data

[0054] This invention employs a Transformer-based multimodal deep learning method to achieve a deep understanding of the semantics of text and image data.

[0055] Text processing is performed using the domain-adaptive AdaptBERT model: the input text is mapped to WordPiece tags by a tokenizer, and then fed into the AdaptBERT model to obtain a context-aware text representation. Based on this, a geographic information corpus is used for domain-adaptive fine-tuning to further improve domain specificity.

[0056] Image processing utilizes the YOLOv5 image processing model to detect various target features such as people, vehicles, and buildings in geographic location images. It also uses Vision Transformer to extract global scene features and combines them with the obtained image features to achieve scene understanding.

[0057] Multimodal data fusion processing uses a cross-attention mechanism, treating text representation as the query and image features as the key and value. It employs a multi-head attention method to calculate the weights of image-text associations, achieving fine-grained semantic alignment between image and text data, accurately locating the association regions between images and text, and providing a basis for cross-modal relationship extraction and knowledge graph construction.

[0058] Relation extraction, based on the weights of graph-text associations, automatically identifies semantic relationships between entities from multimodal data, ultimately transforming them into structured triple facts. Enriching the construction of knowledge graphs, among which, This is the head entity vector in the triplet. This is a triplet relation vector. is the tail entity vector in the triplet.

[0059] The relation extraction model uses a pointer network structure to first identify entities in a given text or image, and then predict the relationship type between entities. The loss function is defined as a weighted sum of category cross-entropy and position regression loss.

[0060] Knowledge graph embedding uses the RotatE model, which represents relations as rotation operations in a complex vector space, and the scoring function is defined as follows:

[0061]

[0062] in, This is the Hadamard product. This model can effectively model various relationship patterns, including symmetric / asymmetric and inverted relationships.

[0063] Step 4: Dynamic Update and Reasoning of Knowledge Graph Based on Time Awareness

[0064] Knowledge graph representation learning employs a time-aware approach to support the dynamic evolution of the graph. It uses an attribute graph model for knowledge storage, where each node includes spatiotemporal, semantic, and state attributes, and edges represent spatiotemporal, semantic, and statistical relationships.

[0065] This invention employs an incremental learning approach for graph updates, where newly added data is aligned and correlated to obtain candidate fact triples. The data is then processed in the quality assessment module and updated in the graph after consistency verification, confidence assessment, and source credibility verification.

[0066] Knowledge graph representation learning uses a time-aware graph neural network (T-GNN), and the node representation update formula is as follows:

[0067]

[0068] in, Represents a node exist The feature vector of the layer, i.e., the updated feature vector. For nodes The set of neighboring nodes, for The nodes in Let the attention weights between node u and node v be determined, taking into account both structural similarity and temporal proximity. for Layer weight matrix, For nodes exist The feature vector representation of the layer, for Layer bias vector.

[0069] The reasoning mechanism is divided into three levels: rule-based reasoning uses SWRL rules for common sense reasoning; embedding-based reasoning predicts implicit relationships based on the knowledge obtained from embedding by aligning different embedding spaces; and path-based reasoning uses random walks to discover the connections between entities.

[0070] Compared to previous methods, this invention effectively improves the fusion effect of geographic information data. Addressing the uncertainty of geographic information data, it establishes a refined probabilistic model, utilizes a probability density function to achieve flexible alignment, and combines deep semantic modeling to achieve the optimal combination of spatiotemporal and semantic evidence. This effectively reduces spatiotemporal bias and improves the accuracy and coverage of multi-source information association. This invention achieves multimodal semantic understanding and association mining for geographic information data. Based on time-aware knowledge graph dynamic updates and reasoning, it can quickly integrate new data sources to enrich the knowledge graph for sudden search and location tasks, achieving global situational awareness and enabling multi-level reasoning based on the knowledge graph. This supports efficient and accurate geographic information data intelligence and improves task response efficiency.

[0071] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for spatiotemporal alignment and semantic association modeling of cross-platform geographic information data, characterized in that, The method includes the following steps: Step 1: Spatiotemporal uncertainty modeling and probabilistic representation to quantify the temporal and spatial uncertainties of multi-source data: For satellite positioning data, elliptical confidence regions are constructed using HDOP values ​​and error parameters; Geographic information is parsed into spatial range using natural language processing technology; a time window model based on data source characteristics is established, taking into account the characteristics of time data sources; a spatiotemporal probability distribution map of data points is established through differentiated spatiotemporal uncertainty modeling methods, and spatiotemporal uncertainty is characterized by probability density functions; Step 2: Combine the alignment decision mechanism with adaptive learning to complete the intelligent flexible alignment fusion of multiple evidences: Based on the calculation of the spatiotemporal correlation of different data objects, the vectors generated by the domain-optimized pre-trained model are used to calculate the cosine similarity to represent the semantic similarity of different data objects. The spatiotemporal correlation and semantic similarity are weighted and fused by a neural network fusion machine to output the comprehensive confidence of the relationship between different data objects. Step 3: Based on multimodal semantic understanding and association mining, conduct deep semantic analysis and real-time association discovery: use a domain adaptive model to realize context-aware text representation of text data; use an image processing model to process image data, and simultaneously realize object detection and scene understanding to obtain image features; use a cross-attention mechanism to realize fine-grained semantic alignment of text and images, and use a pointer network to realize the identification of relationships between entities, and construct knowledge graph fact triples; Step 4: Dynamic updating and reasoning of the knowledge graph, real-time analysis and reasoning: Through an incremental update mechanism, newly added data is processed by the alignment and association module to obtain candidate fact triples. After being screened by the quality assessment module, the data is updated into the knowledge graph. The graph data is processed by a time-aware graph neural network to realize real-time updating and reasoning of the graph, enabling continuous learning and adaptive evolution.

2. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 1, characterized in that, In step one, for satellite positioning data, a probability model based on error propagation theory is used to establish an elliptical confidence region based on the probability distribution of positioning points, according to the horizontal accuracy factor HDOP provided by the positioning equipment used based on the geometric distribution of available satellites. The semi-axis lengths of the ellipse are respectively and ; in The confidence level coefficient is... and The standard deviations of the measurement errors of the positioning device in the east-west and north-south directions are used to accurately describe the probability distribution of the positioning point within the elliptical confidence region through the covariance matrix output by the positioning device: in, Let be the covariance matrix, where for and The correlation coefficient.

3. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 1, characterized in that, In step one, for the geographic information description, natural language processing technology is used to parse out the location description part, and GIS is used to draw a polygon confidence region. The radius of the polygon is determined by the intensity of the descriptive words.

4. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 1, characterized in that, In step one, the time uncertainty modeling method establishes a time window model based on the characteristics of the data source, taking into account the characteristics of the time data source; wherein, the sensor data adopts a normal distribution. Where μ is the timestamp and σ is half of the collection period; manually reported data uses a uniform distribution. ,in, For the time of data collection or recording, Determined based on report type and experience value.

5. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 1, characterized in that, In step two, the spatiotemporal correlation degree is calculated using an improved probability density function overlap integral method: in and The positions are respectively The time is The spatiotemporal probability density functions of the two data objects and Its spatiotemporal domain; Semantic similarity calculation employs a Siamese network structure based on AdaptBERT. Building upon AdaptBERT, it utilizes a large-scale geographic information text corpus for domain-adaptive training to generate geographic information-specific semantic vectors. Cosine similarity is then used to calculate the semantic association strength described by different location data. ; Multi-evidence fusion is achieved by constructing a three-layer fully connected neural network consisting of an input layer, a hidden layer, and an output layer, using a binary cross-entropy loss function. in, The network parameters are obtained by training with historical geographic information data, and the input features are spatiotemporal correlation and semantic similarity. Finally, the overall confidence score within the range of (0, 1) is output, representing the strength of the data association. This association strength is then compared with the decision threshold. Compare the two data points to determine if they are related.

6. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 5, characterized in that, A dynamic learning mechanism is introduced, which continuously learns online through real-time feedback information of the alignment results and dynamically adjusts the decision threshold. : in For the current accuracy rate, For the target accuracy, This is the learning rate, used to adapt to different environmental needs.

7. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 1, characterized in that, In step three Text processing is performed using the domain-adaptive AdaptBERT model: the input text is mapped to WordPiece tags by a word segmenter, and then input into the AdaptBERT model to obtain a context-aware text representation. Based on this, a geographic information corpus is used for domain-adaptive fine-tuning to further improve domain specificity. Image processing utilizes the YOLOv5 image processing model to detect various target features related to geographic information in the image, and uses Vision Transformer to extract global scene features. Combined with the obtained image features, scene understanding is achieved. Multimodal data fusion processing uses a cross-attention mechanism, treating text representation as the query and image features as the key and value. It employs a multi-head attention method to calculate the weights of image-text associations, achieving fine-grained semantic alignment between image and text data, accurately locating the association regions between images and text, and providing a basis for cross-modal relationship extraction and knowledge graph construction.

8. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 7, characterized in that, In step three, relation extraction is based on the weights of graph-text associations, automatically identifying semantic relationships between entities from multimodal data, and ultimately transforming them into structured triple facts. Enriching the construction of knowledge graphs, among which, This is the head entity vector in the triplet. This is a triplet relation vector. is the tail entity vector in the triplet; The relation extraction model uses a pointer network structure to first identify entities in a given text or image, and then predict the relationship type between the entities. The loss function is defined as a weighted sum of category cross-entropy and position regression loss. Knowledge graph embedding uses the RotatE model, which represents relations as rotation operations in a complex vector space, and the scoring function is defined as follows: in, Using the Hadamard product, this model can effectively model various relational schemas.

9. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 1, characterized in that, In step four, incremental learning is used to update the graph. The newly added data is aligned and associated to obtain candidate fact triples. The data is then entered into the quality assessment module and updated into the graph after consistency verification, confidence assessment, and source credibility verification. Knowledge graph representation learning uses a time-aware graph neural network (T-GNN), and the node representation update formula is as follows: in, Represents a node exist The feature vector of the layer, i.e., the updated feature vector. For nodes The set of neighboring nodes, for The nodes in Let the attention weights between node u and node v be determined, taking into account both structural similarity and temporal proximity. for Layer weight matrix, For nodes exist The feature vector representation of the layer, for Layer bias vector.

10. The spatiotemporal alignment and semantic association modeling method for cross-platform geographic information data as described in claim 9, characterized in that, In step four, the reasoning mechanism is divided into three levels: rule-based reasoning uses SWRL rules for common sense reasoning; embedding-based reasoning predicts implicit relationships based on the knowledge obtained from embedding by aligning different embedding spaces; and path-based reasoning uses random walks to discover the connections between entities.

Images