Space-time knowledge graph establishment method and device, equipment and storage medium

Abstract

The invention relates to the technical field of knowledge maps, and discloses a spatio-temporal knowledge map establishment method, device and equipment and a storage medium, multi-source spatio-temporal data is preprocessed, the preprocessed multi-source spatio-temporal data is input, pre-trained and injected into a large language model with spatio-temporal perception ability, and a spatio-temporal knowledge map is established. Outputting an entity with space-time confidence, a relation triple and a space-time trajectory fragment thereof; performing knowledge fusion on an output result by adopting a space-time multi-mode graph attention network to form a space-time knowledge graph; when a new data stream enters, real-time updating of the space-time knowledge graph and entity space-time trajectory recording are carried out; on the spatio-temporal knowledge graph, link prediction and event deduction are carried out by utilizing a spatio-temporal graph neural network, reinforcement learning decision optimization is combined, a spatio-temporal reasoning result and a decision scheme are output, and services are provided for an application system through an API interface; the dynamic spatio-temporal information processing capability is improved, and the knowledge graph updating efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph, in particular to a spatio-temporal knowledge graph establishment method, device and equipment and storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence, big data and geographic information system, as a structured representation method of fusing spatio-temporal information and semantic knowledge, spatio-temporal knowledge graph has become the core support of intelligent decision-making in many fields. The spatio-temporal knowledge graph not only needs to represent the relationship between entities, but also needs to depict the dynamic state and spatial distribution characteristics of entities changing over time. Traditional knowledge graphs represent entities and their relationships in a static structure, but cannot fully capture complex spatio-temporal dynamic information. At the same time, with the popularity of sensor networks and the Internet of Things, massive spatio-temporal data is constantly emerging. How to efficiently process, integrate and utilize these data to build high-quality knowledge graphs has become a key technical problem. SUMMARY

[0003] The present application aims to solve the above problems, and designs a spatio-temporal knowledge graph establishment method, device, equipment and storage medium.

[0004] The first aspect of the present application provides a spatio-temporal knowledge graph establishment method, which comprises: Collecting multi-source spatio-temporal data from sensor networks, geographic information systems, Internet of Things devices and text reports, and preprocessing the multi-source spatio-temporal data, wherein the preprocessing process includes timestamp normalization, spatial coordinate unification and outlier filtering; Inputting the preprocessed multi-source spatio-temporal data into a large language model pre-trained and injected with spatio-temporal perception ability, identifying spatio-temporal entities, extracting spatio-temporal relationships between entities, and outputting entity, relationship triplets with spatio-temporal confidence and their spatio-temporal trajectory fragments; Using a spatio-temporal multi-modal graph attention network to fuse knowledge from the output results, dynamically adjusting the attention weight of message passing between nodes in the graph by calculating the spatial proximity, time series similarity and semantic similarity between entities, and forming a spatio-temporal knowledge graph; When new data streams enter, based on an event-driven incremental update algorithm and combined with a spatio-temporal trajectory tracking algorithm, real-time updating of the spatio-temporal knowledge graph and recording of entity spatio-temporal trajectories are performed; On the spatio-temporal knowledge graph, link prediction and event deduction are performed using a spatio-temporal graph neural network, combined with reinforcement learning decision optimization, and spatio-temporal reasoning results and decision schemes are output, and services are provided to application systems through an API interface.

[0005] Optionally, in the first implementation manner of the first aspect of the present application, the pre-processed multi-source spatio-temporal data is input into the pre-trained large language model with spatio-temporal perception ability, spatio-temporal entities are recognized, spatio-temporal relations between entities are extracted, and entity-relation triplets with spatio-temporal confidence and spatio-temporal trajectory fragments are output, including: obtaining a prompt template including spatio-temporal logic, wherein the prompt template includes an embedded entity time attribute description, a spatial attribute, and a spatio-temporal relation type guide field; calculating the Euclidean distance based on the latitude and longitude between entities and the time interval, and taking entity pairs with a spatial distance and a time interval less than a preset threshold as positive sample pairs, and taking entity pairs with a spatial distance greater than a preset threshold or a time interval greater than a preset threshold as negative sample pairs; introducing spatio-temporal position encoding into the entity and relation label input layer of the large language model, wherein the spatio-temporal position encoding is formed by splicing the latitude and longitude coordinates and the time stamp after being transformed by the sine and cosine function and the Transformer position encoding; inputting the positive sample pairs and the negative sample pairs including the prompt template into the large language model, training the model through a contrastive learning loss function, and enabling the model to learn to distinguish between spatio-temporal proximity and distant entity relations; inputting the pre-processed multi-source spatio-temporal data into the trained large language model, and enabling the large language model to recognize spatio-temporal entities with time and spatial attributes from the input data and extract the time sequence, spatial proximity, and spatio-temporal correlation between entities through the learned spatio-temporal perception ability; calculating the spatio-temporal confidence of each entity-relation triplet based on the entity recognition probability and the relation extraction probability output by the large language model; sorting the entity-relation triplets with spatio-temporal confidence, extracting the spatial position information of each entity at different time points, forming and outputting corresponding spatio-temporal trajectory fragments.

[0006] Optionally, in the second implementation manner of the first aspect of the present application, the calculation of the spatio-temporal confidence of each entity-relation triplet based on the entity recognition probability and the relation extraction probability output by the large language model includes: extracting the probability value output by the large language model to obtain the entity recognition probability and the relation extraction probability between entities in each entity-relation triplet, wherein the entity recognition probability includes the head entity recognition probability and the tail entity recognition probability; obtaining the occurrence frequency and time continuity of the entity in the corresponding time stamp sequence, and calculating the time consistency score; judging whether the spatial position is within a preset geographical range based on the latitude and longitude coordinates of the entity, and calculating whether the spatial movement trajectory of the entity exceeds a preset distance range to obtain the spatial consistency score; Based on the time consistency score and the space consistency score, a spatiotemporal consistency comprehensive score is calculated by weighted fusion; Determine the weight coefficients of entity recognition, relationship extraction and spatiotemporal consistency, and calculate the spatiotemporal confidence of each entity, relationship triple : Among them, represents the head entity recognition weight coefficient, represents the head entity recognition probability, represents the tail entity recognition weight coefficient, represents the tail entity recognition probability, represents the relationship extraction weight coefficient, represents the relationship extraction probability, represents the spatiotemporal consistency weight coefficient, represents the spatiotemporal consistency comprehensive score.

[0007] Optionally, in the third implementation manner of the first aspect of the present application, the output result is fused by using the spatiotemporal multi-modal graph attention network, the attention weight of message transmission between nodes in the graph is dynamically adjusted by calculating the spatial proximity, time sequence similarity and semantic similarity between entities, and a spatiotemporal knowledge graph is formed, comprising: Based on the entity, relationship triple and spatiotemporal trajectory segment with spatiotemporal confidence, a spatiotemporal multi-modal graph structure is constructed, the entity is taken as a graph node, the spatiotemporal relationship between entities is taken as an initial edge of the graph, the node attribute includes the spatiotemporal feature and semantic feature of the entity, and the edge attribute includes the initial relationship type; The spatial proximity between entities is calculated by using the Euclidean distance : Among them, , are the longitude and latitude of the entity , , are the longitude and latitude of the entity ; The time sequence similarity between entities is calculated by using the DTW algorithm, the optimal matching path between the timestamp sequence of the entity and the timestamp sequence of the entity is determined, and the cumulative distance is calculated; The semantic similarity between entities is calculated by using the cosine similarity : Among them, , are the semantic vectors of the entity , semantic embedding vectors of the entities; Based on the normalized inter-entity spatial proximity, inter-entity time series similarity and inter-entity semantic similarity, the attention weight of message passing between graph nodes is dynamically adjusted by weighted fusion, and the final attention weight is obtained by normalization through a softmax function. According to the final attention weight, the neighbor node features are aggregated to update the feature vector of the current node, and the attention weight calculation and message passing process are repeated. When the iteration threshold is reached, the updated node features, the spatio-temporal relationship between nodes and the final attention weight are integrated to form a spatio-temporal knowledge graph.

[0008] Optionally, in the fourth implementation manner of the first aspect of the present application, when the new data stream enters, the real-time updating of the spatio-temporal knowledge graph and the entity spatio-temporal trajectory recording are performed based on the event-driven incremental updating algorithm and combined with the spatio-temporal trajectory tracking algorithm, including: Based on the spatio-temporal features and attribute features in the new data stream, the event type is divided, and the identified event is assigned a corresponding priority according to the preset event priority rule; The spatio-temporal similarity and semantic similarity between the entity in the new data stream and the entity in the spatio-temporal knowledge graph are calculated, and when the spatio-temporal similarity and semantic similarity are both higher than the preset threshold, it is determined as the same entity, otherwise it is determined as a new entity; A graph node is created for the new entity, the node attributes are initialized, the initial relationship edges with the associated entities are established, and the new node is added to the spatio-temporal knowledge graph. If the attribute is updated, the attribute value corresponding to the existing node is replaced, if the relationship is changed, the corresponding relationship edge is added or deleted, and the edge attribute is adjusted; The timestamp and latitude and longitude coordinates of the new entity are extracted and added to the spatio-temporal trajectory to obtain the updated entity spatio-temporal trajectory; The spatial movement distance and time interval between the current trajectory point and the previous trajectory point are calculated, and the continuity of the updated entity spatio-temporal trajectory is verified; Based on the latest spatio-temporal trajectory and attribute information of the entity, the spatio-temporal index of the graph is updated, and the update log is recorded.

[0009] Optionally, in the fifth implementation manner of the first aspect of the present application, the identified event is assigned a corresponding priority according to the preset event priority rule, including: The high-priority event triggers real-time updating immediately, the medium-priority event triggers quasi-real-time updating, and the low-priority event is temporarily stored in a batch updating queue and waits for a preset time window to trigger batch updating. Quasi-real-time updating is an updating mode between real-time updating and batch updating.

[0010] Optionally, in the sixth implementation form of the first aspect of the present application, the link prediction and event deduction on the spatio-temporal knowledge graph are performed by using a spatio-temporal graph neural network, a decision optimization is combined with reinforcement learning, a spatio-temporal reasoning result and a decision scheme are output, and services are provided to an application system through an API interface, including: The spatio-temporal knowledge graph is input into a spatio-temporal graph neural network including a spatial graph convolution layer and a time series processing layer, the spatial graph convolution layer aggregates spatial features, and the time series processing layer processes time series features of entities by using a gated recurrent unit to predict a spatial diffusion range of an event at a future time point and output a spatio-temporal evolution path of the event; A link existence energy value is calculated based on node embedding vectors and relation embedding vectors generated in a spatio-temporal graph neural network processing process to obtain a link prediction result: A state space is defined as a state of a current spatio-temporal knowledge graph, an action space is defined as a possible decision scheme, and a reward function is defined as a target achievement degree after an action is performed; An agent is trained by using a DDPG algorithm, an actor network outputs a deterministic action based on a current state, and a critic network evaluates an action value; The link prediction result and the spatio-temporal evolution path are input into the trained agent to output an optimal decision scheme; The link prediction result, the spatio-temporal evolution path, and the optimal decision scheme are integrated, are pushed to an application system through an API interface, and API interface call logs are recorded.

[0011] The second aspect of the present application provides a spatio-temporal knowledge graph establishment device, and the spatio-temporal knowledge graph establishment device includes: A spatio-temporal data acquisition and standardization module is configured to acquire multi-source spatio-temporal data from a sensor network, a geographic information system, an Internet of Things device, and a text report, and to pre-process the multi-source spatio-temporal data, wherein the pre-processing process includes timestamp normalization, spatial coordinate unification, and abnormal value filtering; A large model spatio-temporal semantic understanding and entity relation extraction module is configured to input the pre-processed multi-source spatio-temporal data into a large language model that is pre-trained and injected with spatio-temporal perception capability, to identify spatio-temporal entities and extract spatio-temporal relations between the entities, and to output entity, relation triplets with spatio-temporal confidence and spatio-temporal trajectory segments thereof; A spatio-temporal entity fusion and knowledge integration module is configured to perform knowledge fusion on the output results by using a spatio-temporal multi-modal graph attention network, to dynamically adjust attention weights of message transmission between nodes in a graph by calculating spatial proximity, time series similarity, and semantic similarity between entities, and to form a spatio-temporal knowledge graph; A dynamic updating and spatio-temporal evolution module is configured to, when new data flows enter, perform real-time updating of the spatio-temporal knowledge graph and recording of spatio-temporal trajectories of entities based on an event-driven incremental updating algorithm and in combination with a spatio-temporal trajectory tracking algorithm. The spatio-temporal reasoning and application interface module is used for link prediction and event deduction on the spatio-temporal knowledge graph by using a spatio-temporal graph neural network, combining reinforcement learning decision optimization, outputting spatio-temporal reasoning results and decision schemes, and providing services to an application system through an API interface.

[0012] The third aspect of the present application provides a spatio-temporal knowledge graph establishment device, comprising a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to enable the spatio-temporal knowledge graph establishment device to perform the steps of the spatio-temporal knowledge graph establishment method according to any one of the above.

[0013] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium storing instructions, the instructions being executed by a processor to implement the steps of the spatio-temporal knowledge graph establishment method according to any one of the above.

[0014] In the technical solution provided by the present application, multi-source spatio-temporal data from a sensor network, a geographic information system, an Internet of Things device and a text report is collected, and the multi-source spatio-temporal data is preprocessed; the preprocessed multi-source spatio-temporal data is input into a large language model pre-trained and injected with spatio-temporal perception capability, spatio-temporal entities are identified, spatio-temporal relationships between entities are extracted, and entity, relationship triplets with spatio-temporal confidence and their spatio-temporal trajectory segments are output; the output results are knowledge fused by using a spatio-temporal multi-modal graph attention network, the attention weights of message transmission between nodes in the graph are dynamically adjusted by calculating the spatial proximity, time sequence similarity and semantic similarity between entities, and a spatio-temporal knowledge graph is formed; when new data streams enter, an event-driven incremental update algorithm is used, combined with a spatio-temporal trajectory tracking algorithm, to update the spatio-temporal knowledge graph in real time and record the spatio-temporal trajectories of entities; on the spatio-temporal knowledge graph, link prediction and event deduction are performed by using a spatio-temporal graph neural network, combined with reinforcement learning decision optimization, to output spatio-temporal reasoning results and decision schemes, and services are provided to an application system through an API interface; the present application improves the dynamic spatio-temporal information processing capability, realizes accurate modeling of entities and relationships changing with time and space, enhances the multi-source heterogeneous data fusion capability, guarantees the integrity and consistency of the knowledge graph, improves the knowledge graph update efficiency, reduces data conflicts and improves real-time performance, optimizes the adaptability of the large model in the spatio-temporal field, enhances the semantic reasoning and relationship discovery capability, and the event-triggered incremental update algorithm reduces the update delay, meeting the real-time scene requirements of intelligent transportation, emergency response and the like. BRIEF DESCRIPTION OF DRAWINGS

[0015] Various other advantages and benefits will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments, and are not meant to limit the present application.

[0016] Figure 1 The first embodiment schematic diagram of the spatio-temporal knowledge graph establishment method provided by the embodiment of the application is shown in the figure. Figure 2 The second embodiment schematic diagram of the spatio-temporal knowledge graph establishment method provided by the embodiment of the application is shown in the figure. Figure 3 The third embodiment schematic diagram of the spatio-temporal knowledge graph establishment method provided by the embodiment of the application is shown in the figure. Figure 4 The structure schematic diagram of the spatio-temporal knowledge graph establishment device provided by the embodiment of the application is shown in the figure. Figure 5 The structure schematic diagram of the spatio-temporal knowledge graph establishment device provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0017] The terms "first", "second", "third", "fourth" and the like in the description, claims, and drawings of the application, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms in the description is not meant to limit the scope of the application to the particular order or sequence, unless otherwise specifically stated. It is to be understood that where the application is indicated to include process steps, or methods, or procedures, or steps, or units, or devices, or products, or the like, the process steps, or methods, or procedures, or steps, or units, or devices, or products, or the like, can be carried out in any order, unless otherwise specifically stated. Furthermore, the term "comprising" or "containing" and variations thereof as used in the description and throughout the claims, are meant to cover the process steps, or methods, or procedures, or steps, or units, or devices, or products, or the like, without open or close ended, unless otherwise specifically stated.

[0018] For the sake of understanding, the specific flow of the embodiment of the application is described below, please refer to Figure 1 The first embodiment schematic diagram of the spatio-temporal knowledge graph establishment provided by the embodiment of the application is shown in the figure, and the method specifically includes the following steps: Step 101, collecting multi-source spatio-temporal data from a sensor network, a geographic information system, an Internet of Things device and a text report, and pre-processing the multi-source spatio-temporal data, wherein the pre-processing process includes timestamp normalization, spatial coordinate unification and outlier filtering; In this embodiment, a dynamic time warping algorithm is used to perform time alignment. First, the timestamp of a core data source is selected as a reference benchmark. Then, the time difference distribution between the time series of other data sources and the reference benchmark is calculated. The optimal matching relationship of time points of each data source is found through dynamic programming. The timestamps of non-reference data sources are corrected according to the matching results, so that the relevant data of the same event or the same monitoring object are kept synchronized in the time dimension. A spatial projection transformation algorithm is used to perform spatial alignment. First, the original coordinate system of each data source is identified. Then, the corresponding projection transformation model is selected according to the coordinate system parameters. The spatial coordinates of all data sources are uniformly converted into the preset target coordinates to ensure that the spatial location description of geographic entities is consistent. An isolation forest model is constructed for anomaly data screening. First, a portion of samples is randomly selected from the normalized spatiotemporal data as the training set. Based on the spatiotemporal distribution characteristics of the data, multiple isolation trees are constructed. Each isolation tree gradually isolates the training samples by randomly selecting features and split points, forming different leaf nodes. All preprocessed data is input into the trained isolation forest model, and the path length of each data sample in all isolation trees is calculated. Data with path lengths significantly shorter than normal samples are considered anomalies. The marked anomalies are stored separately for verification. Set a data integrity threshold, and check each data record field by field. Remove data records with a missing rate of core fields exceeding the threshold, such as timestamps, spatial coordinates, and key monitoring parameters. For the remaining data, identify completely duplicate data and highly similar data by comparing spatiotemporal coordinates and core attribute information, such as data with spatial coordinate errors within a preset range and completely identical attribute information. For completely duplicate data, directly retain one record. For highly similar data, retain the record with the highest credibility based on the credibility priority of the data source, thus completing the removal of redundant data.

[0019] Step 102: Input the preprocessed multi-source spatiotemporal data into a large language model that has been pre-trained and injected with spatiotemporal awareness capabilities, identify spatiotemporal entities, extract spatiotemporal relationships between entities, and output entities, relation triples and their spatiotemporal trajectory fragments with spatiotemporal confidence. Step 103: The output results are fused using a spatiotemporal multimodal graph attention network. By calculating the spatial proximity, temporal series similarity, and semantic similarity between entities, the attention weights of message passing between nodes in the graph are dynamically adjusted to form a spatiotemporal knowledge graph. In this embodiment, a spatiotemporal multimodal graph structure is constructed based on entity and relation triples with spatiotemporal confidence and spatiotemporal trajectory fragments. Entities are used as graph nodes, and the spatiotemporal relationships between entities are used as initial edges of the graph. Node attributes include entity spatiotemporal features and semantic features, and edge attributes include initial relation type. Calculating spatial proximity between entities using Euclidean distance : in, , For entities latitude and longitude , For entities latitude and longitude; The DTW algorithm is used to calculate the time series similarity between entities and determine the entities. Timestamp sequences and entities Find the optimal matching path between timestamp sequences and calculate the cumulative distance; Cosine similarity is used to calculate semantic similarity between entities. : in, , Entities , semantic embedding vector; Based on the normalized spatial proximity between entities, temporal similarity between entities, and semantic similarity between entities, the attention weights for message passing between graph nodes are dynamically adjusted through weighted fusion, and then the final attention weights are obtained by normalization using the softmax function. Based on the final attention weight, the features of neighboring nodes are aggregated, the feature vector of the current node is updated, and the attention weight calculation and message passing process is repeated. When the iteration threshold is reached, the updated node features, the spatiotemporal relationships between nodes, and the final attention weight are integrated to form a spatiotemporal knowledge graph.

[0020] Step 104: When a new data stream enters, the spatiotemporal knowledge graph is updated in real time and the spatiotemporal trajectory of entities is recorded based on the event-driven incremental update algorithm and the spatiotemporal trajectory tracking algorithm. Step 105: On the spatiotemporal knowledge graph, use the spatiotemporal graph neural network to perform link prediction and event inference, combine reinforcement learning decision optimization, output spatiotemporal reasoning results and decision schemes, and provide services to application systems through API interfaces.

[0021] In this embodiment, the spatiotemporal knowledge graph is input into a spatiotemporal graph neural network, which includes a spatial graph convolutional layer and a time series processing layer. The spatial graph convolutional layer aggregates spatial features, and the time series processing layer uses gated recurrent units to process the time series features of entities, predicting the spatial diffusion range of an event at a future time point and outputting the spatiotemporal evolution path of the event. Based on the node embedding vectors and relation embedding vectors generated during the processing of the spatiotemporal graph neural network, the energy value of the link existence is calculated to obtain the link prediction result. The state space is defined as the current state of the spatiotemporal knowledge graph, the action space is the possible decision scheme, and the reward function is the degree of goal achievement after executing the action. The agent is trained using the DDPG algorithm. The actor network outputs deterministic actions based on the current state, and the critic network evaluates the value of the actions. The link prediction result and the spatiotemporal evolution path are input into the trained agent to output the optimal decision scheme. The link prediction result, the spatiotemporal evolution path, and the optimal decision scheme are integrated and pushed to the application system through an API interface, and the API interface call log is recorded.

[0022] Please see Figure 2 A schematic diagram of a second embodiment of the spatiotemporal knowledge graph establishment provided by the present invention, the method including: Step 201: Obtain the prompt word template including spatiotemporal logic, wherein the prompt word template includes embedded entity time attribute description, spatial attribute, and spatiotemporal relationship type guiding field; In this embodiment, the entity time attribute description field explicitly requires the marking of the precise timestamp of the event, the time range, or the description of the temporal relationship; the spatial attribute field requires the explicit marking of the entity's latitude and longitude coordinates, geographical region description, or spatial relationship keywords; the spatiotemporal relationship type guidance field presets common spatiotemporal relationship category tags, such as spatiotemporal association, spatial proximity association, temporal sequence association, and spatiotemporal causal association.

[0023] Step 202: Calculate the Euclidean distance and time interval between entities based on latitude and longitude. Entity pairs with both spatial distance and time interval less than a preset threshold are taken as positive sample pairs, and entity pairs with spatial distance or time interval greater than a preset threshold are taken as negative sample pairs. In this embodiment, based on the latitude and longitude coordinates of entities and graph entities in the new data stream, the geographic straight-line distance between two points is calculated using the Euclidean distance formula to obtain a spatial distance quantification value. Then, the time interval is calculated: the precise timestamps corresponding to the two types of entities are extracted, and the absolute time difference between the two is calculated to obtain a time interval quantification value. Based on historical data and application scenario requirements, a spatial distance threshold and a time interval threshold are preset. When the spatial distance of a pair of entities is less than the spatial threshold and the time interval is less than the time threshold, it is determined to be a positive sample pair that is spatiotemporally close. When the spatial distance of a pair of entities is greater than the spatial threshold or the time interval is greater than the time threshold, it is determined to be a negative sample pair that is spatiotemporally distant. The two types of sample pairs together constitute the comparison sample set for model training, ensuring that the training data covers different spatiotemporally related scenarios.

[0024] Step 203: Introduce spatiotemporal location encoding for entity and relation labels in the input layer of the large language model. The spatiotemporal location encoding is formed by concatenating Transformer location encoding with latitude and longitude coordinates and timestamps after sine and cosine function transformation. In this embodiment, based on the position of the data in the input sequence and the model embedding dimension, the sequence position features are generated through sine and cosine functions. Then, sine and cosine transformations are performed on the latitude and longitude coordinates and timestamps of the entities, respectively. The transformation process uses the same embedding dimension as the Transformer position encoding. After standardizing the numerical ranges of latitude and longitude and the numerical ranges of timestamps, they are substituted into the sine and cosine functions to generate independent spatial feature codes and temporal feature codes. The standard Transformer position code, the spatial code after latitude and longitude transformation, and the temporal code after timestamp transformation are concatenated according to the embedding dimension direction to form a complete spatiotemporal position code. The spatiotemporal position code not only retains the sequence information but also incorporates the spatial position and temporal attributes of the entities, enabling the model to synchronously perceive the spatiotemporal dimension features when processing data.

[0025] Step 204: Input the positive and negative sample pairs, including the prompt word templates, into the large language model, and train the model through the contrastive learning loss function so that the model can learn to distinguish between spatiotemporally close and distant entity relationships. In this embodiment, the loss function constructs the loss value by calculating the similarity of the embedding vectors of positive sample pairs and the similarity of the embedding vectors of negative sample pairs. During the training process, the model parameters are continuously optimized so that the embedding vector distance of positive sample pairs is continuously reduced and the embedding vector distance of negative sample pairs is continuously increased. Through multiple rounds of iterative training, the model gradually learns the correlation rules in the spatiotemporal dimension and can accurately distinguish whether different entities are spatiotemporally close or spatiotemporally distant, thus possessing a stable spatiotemporal perception capability.

[0026] Step 205: Input the preprocessed multi-source spatiotemporal data into the trained large language model. The large language model uses its learned spatiotemporal perception ability to identify spatiotemporal entities with time and space attributes from the input data and extract the temporal sequence, spatial proximity, and spatiotemporal relationship between entities. In this embodiment, the preprocessed multi-source spatiotemporal data is organized according to the model input format. The data already includes latitude and longitude in a unified coordinate system, normalized timestamps, and structured attribute descriptions to ensure consistency with the training data format. Subsequently, the organized data is input into the trained large language model. The model will simultaneously call upon its learned spatiotemporal perception capabilities and its own semantic understanding capabilities: at the entity recognition level, not only is the core identification information of the entity extracted, but its corresponding time and spatial attributes are also associated to form spatiotemporal entities with spatiotemporal labels; at the relation extraction level, based on the spatiotemporal encoding and semantic embedding of entities, the temporal sequence relationship, spatial proximity relationship, and spatiotemporal association relationship between entities are automatically identified, and finally, a structured set of spatiotemporal entities and corresponding relationships is output.

[0027] Step 206: Based on the entity recognition probability and relation extraction probability output by the large language model, calculate the spatiotemporal confidence of each entity and relation triplet; Step 207: Organize entity and relation triples with spatiotemporal confidence, extract the spatial location information of each entity at different time points, and form and output the corresponding spatiotemporal trajectory fragments.

[0028] In this embodiment, the probability values ​​output by the large language model are extracted, and the entity recognition probability and the relationship extraction probability between entities are obtained for each entity and relation triple. The entity recognition probability includes the head entity recognition probability and the tail entity recognition probability. Obtain the frequency of occurrence and temporal continuity of entities in the corresponding timestamp sequence, and calculate the temporal consistency score; Based on the latitude and longitude coordinates of an entity, determine whether its spatial location is within a preset geographical range, and calculate whether the entity's spatial movement trajectory exceeds a preset distance range to obtain a spatial consistency score. Based on temporal consistency scores and spatial consistency scores, a weighted fusion calculation is performed to obtain a comprehensive temporal consistency score. Determine the weighting coefficients for entity recognition, relation extraction, and spatiotemporal consistency, and calculate the spatiotemporal confidence of each entity and relation triplet. : in, This represents the weighting coefficient for head entity recognition. Indicates the probability of head entity recognition. This represents the tail entity recognition weight coefficient. Indicates the probability of tail entity recognition. This represents the weighting coefficient for relation extraction. This represents the probability of relation extraction. This represents the spatiotemporal consistency weighting coefficient. This represents the overall score for spatiotemporal consistency.

[0029] Please see Figure 3 The third embodiment of the spatiotemporal knowledge graph establishment provided by the present invention is illustrated in the following diagram: Step 301: Classify event types based on the spatiotemporal and attribute features in the new data stream, and assign corresponding priorities to the identified events according to the preset event priority rules; In this embodiment, high-priority events trigger real-time updates immediately, medium-priority events trigger near-real-time updates, and low-priority events are temporarily stored in a batch update queue, waiting for a preset time window to trigger batch updates. The near-real-time update is an update mode between real-time updates and batch updates. After a medium-priority event is triggered, the system does not immediately execute the update operation, but allows a short buffer time. Medium-priority events of the same type or in the same area can be lightly aggregated before being centrally updated. This mode balances the timeliness of updates with the consumption of system resources and is suitable for scenarios that do not require emergency response but need to ensure the synchronization of knowledge graph data in a short period of time, such as updates of traffic flow on ordinary roads and fine-tuning of environmental parameters in non-key areas.

[0030] Step 302: Calculate the spatiotemporal similarity and semantic similarity between entities in the new data stream and entities in the spatiotemporal knowledge graph. When both the spatiotemporal similarity and semantic similarity are higher than the preset threshold, they are determined to be the same entity; otherwise, they are determined to be newly added entities. Step 303: Create graph nodes for the new entities, initialize node attributes, establish initial relationship edges with related entities, and add the new nodes to the spatiotemporal knowledge graph. If the attributes are updated, replace the corresponding attribute values ​​of the existing nodes. If the relationships change, add or delete the corresponding relationship edges and adjust the edge attributes. In this embodiment, for entities identified as new, a unique node ID is first generated, and then node attributes are initialized. These attributes cover categories such as spatiotemporal basic features, semantic features, and data source features. Simultaneously, based on the association information contained in the new data stream, such as spatial proximity and semantic association with other known entities, and combined with semantic matching results, initial relationship edges are established between the new node and existing associated entities in the graph. The type of relationship edge is determined according to the actual association logic. Finally, the new node and its associated edges are formally incorporated into the spatiotemporal knowledge graph. For cases where the new data stream contains attribute update content, only the corresponding attribute values ​​in the existing node are replaced. If changes in relationships between entities are involved, when new data shows a new association, a corresponding new relationship edge is created and its attributes are initialized, such as relationship confidence and association time range. When data shows that the original relationship has become invalid, such as when a change in entity status causes the relationship to terminate, the corresponding relationship edge is deleted, and the relevant attributes of the remaining relationship edges are adjusted, such as updating the effective time range of the relationship and correcting the relationship confidence.

[0031] Step 304: Extract the timestamp and latitude / longitude coordinates of the newly added entity and add them to the spatiotemporal trajectory to obtain the updated entity spatiotemporal trajectory; Step 305: Calculate the spatial movement distance and time interval between the current trajectory point and the previous trajectory point, and perform continuity verification on the updated entity spatiotemporal trajectory; In this embodiment, the current trajectory point and the previous trajectory point in the updated spatiotemporal trajectory are located, and then two key indicators are calculated: the spatial movement distance is calculated based on the latitude and longitude coordinates of the two points using the Euclidean distance formula to determine the actual geographical distance; the time interval is obtained by subtracting the timestamp of the previous trajectory point from the timestamp of the current trajectory point, with the unit consistent with the timestamp precision; a reasonable range threshold is preset according to the type of entity, such as the maximum movement speed threshold for vehicle entities, the normal movement speed range for personnel entities, and the position fluctuation threshold for static entities, etc. The calculated spatial movement distance / time interval is compared with the preset threshold. If it is within a reasonable range, the trajectory is determined to be continuous and valid, and the trajectory point is retained; if it exceeds a reasonable range, it is marked as an abnormal trajectory point and is not included in the formal trajectory sequence.

[0032] Step 306: Update the spatiotemporal index of the map based on the latest spatiotemporal trajectory and attribute information of the entity, and record the update log.

[0033] In this embodiment, the index system of the spatiotemporal knowledge graph is synchronously updated based on the latest spatiotemporal trajectory and updated attribute information of the entities. The index types involved include time range index, spatial region index, and spatiotemporal joint index. During the update process, a strategy of adding first and then deleting is adopted, creating new index entries first and then deleting invalid old indexes to avoid query interruption. At the same time, the system automatically generates detailed update logs, which include a unique log identifier, update execution time, module that performed the update, a list of entity IDs involved, a comparison of attribute changes before and after, details of relationship edge adjustments, trajectory update records, etc. The logs are stored in a structured format in a designated database.

[0034] Please see Figure 4 A schematic diagram of a structure for establishing a spatiotemporal knowledge graph provided in this embodiment of the invention. The system includes: The spatiotemporal data acquisition and standardization module 401 is used to acquire multi-source spatiotemporal data from sensor networks, geographic information systems, Internet of Things devices and text reports, and to preprocess the multi-source spatiotemporal data, wherein the preprocessing process includes timestamp normalization, spatial coordinate unification and outlier filtering. The large model spatiotemporal semantic understanding and entity relationship extraction module 402 is used to input pre-processed multi-source spatiotemporal data into a large language model that has been pre-trained and injected with spatiotemporal perception capabilities, identify spatiotemporal entities, extract spatiotemporal relationships between entities, and output entities, relationship triples and their spatiotemporal trajectory fragments with spatiotemporal confidence. The spatiotemporal entity fusion and knowledge integration module 403 is used to perform knowledge fusion on the output results using a spatiotemporal multimodal graph attention network. By calculating the spatial proximity, temporal series similarity and semantic similarity between entities, the attention weight of message passing between nodes in the graph is dynamically adjusted to form a spatiotemporal knowledge graph. The dynamic update and spatiotemporal evolution module 404 is used to perform real-time updates of the spatiotemporal knowledge graph and record the spatiotemporal trajectory of entities based on an event-driven incremental update algorithm and a spatiotemporal trajectory tracking algorithm when new data streams enter. The spatiotemporal reasoning and application interface module 405 is used to perform link prediction and event inference on the spatiotemporal knowledge graph using spatiotemporal graph neural networks, combined with reinforcement learning decision optimization, to output spatiotemporal reasoning results and decision schemes, and to provide services to application systems through API interfaces.

[0035] above Figure 4 The spatiotemporal knowledge graph establishment device in this embodiment of the invention is described in detail from the perspective of modular functional entities. The spatiotemporal knowledge graph establishment device in this embodiment of the invention is described in detail from the perspective of hardware processing.

[0036] Figure 5This is a schematic diagram of a spatiotemporal knowledge graph building device provided in an embodiment of the present invention. The spatiotemporal knowledge graph building device 600 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 610 (e.g., one or more processors) and a memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) for storing application programs 633 or data 632. The memory 620 and storage media 630 can be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the spatiotemporal knowledge graph building device 600. Furthermore, the processor 610 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the spatiotemporal knowledge graph building device 600 to implement the method provided in the above embodiment.

[0037] The spatiotemporal knowledge graph building device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input / output interfaces 660, and / or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 5 The spatiotemporal knowledge graph building device structure shown does not constitute a limitation on the computer device provided by the present invention. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0038] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the various steps of the spatiotemporal knowledge graph establishment method provided in the above embodiments.

[0039] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described equipment or apparatus / unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0040] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0041] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for establishing a spatiotemporal knowledge graph, characterized in that, The method for establishing the spatiotemporal knowledge graph includes: Multi-source spatiotemporal data from sensor networks, geographic information systems, Internet of Things devices, and text reports are collected, and the multi-source spatiotemporal data is preprocessed, including timestamp normalization, spatial coordinate unification, and outlier filtering. The preprocessed multi-source spatiotemporal data is input into a large language model that has been pre-trained and injected with spatiotemporal awareness capabilities to identify spatiotemporal entities, extract spatiotemporal relationships between entities, and output entities, relation triples and their spatiotemporal trajectory fragments with spatiotemporal confidence. The output results are fused using a spatiotemporal multimodal graph attention network. By calculating the spatial proximity, temporal series similarity, and semantic similarity between entities, the attention weights of message passing between nodes in the graph are dynamically adjusted to form a spatiotemporal knowledge graph. When new data streams arrive, an event-driven incremental update algorithm, combined with a spatiotemporal trajectory tracking algorithm, is used to update the spatiotemporal knowledge graph in real time and record the spatiotemporal trajectories of entities. On the spatiotemporal knowledge graph, spatiotemporal graph neural networks are used for link prediction and event inference, combined with reinforcement learning decision optimization, to output spatiotemporal reasoning results and decision schemes, and provide services to application systems through API interfaces.

2. The method for establishing a spatiotemporal knowledge graph as described in claim 1, characterized in that, The process involves inputting pre-processed multi-source spatiotemporal data into a pre-trained large language model with spatiotemporal awareness capabilities to identify spatiotemporal entities, extract spatiotemporal relationships between entities, and output entities, relation triples, and their spatiotemporal trajectory fragments with spatiotemporal confidence scores, including: Obtain a prompt word template that includes spatiotemporal logic, wherein the prompt word template includes embedded entity time attribute description, spatial attribute, and spatiotemporal relationship type guiding field; Calculate the Euclidean distance and time interval between entities based on latitude and longitude. Entity pairs with both spatial distance and time interval less than a preset threshold are regarded as positive sample pairs, and entity pairs with spatial distance or time interval greater than a preset threshold are regarded as negative sample pairs. In the input layer of the large language model, spatiotemporal location encoding is introduced for entity and relation labels. The spatiotemporal location encoding is formed by concatenating Transformer location encoding with latitude and longitude coordinates and timestamps after sine and cosine function transformation. Positive and negative sample pairs, including prompt word templates, are input into a large language model. The model is trained by a contrastive learning loss function, enabling the model to learn to distinguish between spatiotemporally close and distant entity relationships. The preprocessed multi-source spatiotemporal data is input into the trained large language model. The large language model uses its learned spatiotemporal perception to identify spatiotemporal entities with time and space attributes from the input data and extract the temporal sequence, spatial proximity, and spatiotemporal relationship between entities. Based on the entity recognition probability and relation extraction probability output by the large language model, calculate the spatiotemporal confidence of each entity and relation triplet; Organize entity and relation triples with spatiotemporal confidence, extract the spatial location information of each entity at different time points, and form and output the corresponding spatiotemporal trajectory fragments.

3. The spatiotemporal knowledge graph construction method as described in claim 2, characterized in that, The calculation of the spatiotemporal confidence of each entity and relation triple based on the entity recognition probability and relation extraction probability output by the large language model includes: Extract the probability values ​​output by the large language model, and obtain the entity recognition probability and the relationship extraction probability between entities in each entity and relation triplet. The entity recognition probability includes the head entity recognition probability and the tail entity recognition probability. Obtain the frequency of occurrence and temporal continuity of entities in the corresponding timestamp sequence, and calculate the temporal consistency score; Based on the latitude and longitude coordinates of an entity, determine whether its spatial location is within a preset geographical range, and calculate whether the entity's spatial movement trajectory exceeds a preset distance range to obtain a spatial consistency score. Based on temporal consistency scores and spatial consistency scores, a weighted fusion calculation is performed to obtain a comprehensive temporal consistency score. Determine the weighting coefficients for entity recognition, relation extraction, and spatiotemporal consistency, and calculate the spatiotemporal confidence of each entity and relation triplet. : in, This represents the head entity recognition weight coefficient. Indicates the probability of head entity recognition. This represents the tail entity recognition weight coefficient. Indicates the probability of tail entity recognition. This represents the weighting coefficient for relation extraction. This represents the probability of relation extraction. This represents the spatiotemporal consistency weighting coefficient. This represents the overall score for spatiotemporal consistency.

4. The method for establishing a spatiotemporal knowledge graph as described in claim 1, characterized in that, The output results are then fused using a spatiotemporal multimodal graph attention network. By calculating spatial proximity, temporal similarity, and semantic similarity between entities, the attention weights for message passing between nodes in the graph are dynamically adjusted to form a spatiotemporal knowledge graph, including: Based on entity and relation triples with spatiotemporal confidence and spatiotemporal trajectory fragments, a spatiotemporal multimodal graph structure is constructed. Entities are used as graph nodes, and spatiotemporal relationships between entities are used as initial edges of the graph. Node attributes include entity spatiotemporal features and semantic features, and edge attributes include initial relation type. Calculating spatial proximity between entities using Euclidean distance : in, , For entities latitude and longitude , For entities latitude and longitude; The DTW algorithm is used to calculate the time series similarity between entities and determine the entities. Timestamp sequences and entities Find the optimal matching path between timestamp sequences and calculate the cumulative distance; Cosine similarity is used to calculate semantic similarity between entities. : in, , Entities , semantic embedding vector; Based on the normalized spatial proximity between entities, temporal similarity between entities, and semantic similarity between entities, the attention weights for message passing between graph nodes are dynamically adjusted through weighted fusion, and then the final attention weights are obtained by normalization using the softmax function. Based on the final attention weight, the features of neighboring nodes are aggregated, the feature vector of the current node is updated, and the attention weight calculation and message passing process is repeated. When the iteration threshold is reached, the updated node features, the spatiotemporal relationships between nodes, and the final attention weight are integrated to form a spatiotemporal knowledge graph.

5. The method for establishing a spatiotemporal knowledge graph as described in claim 1, characterized in that, When new data streams arrive, an event-driven incremental update algorithm, combined with a spatiotemporal trajectory tracking algorithm, is used to perform real-time updates of the spatiotemporal knowledge graph and record the spatiotemporal trajectories of entities, including: Event types are classified based on the spatiotemporal and attribute features in the new data stream, and corresponding priorities are assigned to the identified events according to the preset event priority rules. Calculate the spatiotemporal similarity and semantic similarity between entities in the new data stream and entities in the spatiotemporal knowledge graph. If both the spatiotemporal similarity and semantic similarity are higher than a preset threshold, they are determined to be the same entity; otherwise, they are determined to be newly added entities. Create graph nodes for new entities, initialize node attributes, establish initial relationship edges with associated entities, and add new nodes to the spatiotemporal knowledge graph. If attributes are updated, replace the corresponding attribute values ​​of existing nodes. If relationships change, add or delete corresponding relationship edges and adjust edge attributes. Extract the timestamp and latitude / longitude coordinates of the newly added entity and add them to the spatiotemporal trajectory to obtain the updated spatiotemporal trajectory of the entity; Calculate the spatial movement distance and time interval between the current trajectory point and the previous trajectory point, and perform continuity verification on the updated entity spatiotemporal trajectory; The spatiotemporal index of the map is updated based on the latest spatiotemporal trajectory and attribute information of the entities, and the update log is recorded.

6. The spatiotemporal knowledge graph construction method as described in claim 5, characterized in that, The step of assigning corresponding priorities to the identified events according to preset event priority rules includes: High-priority events trigger real-time updates immediately, medium-priority events trigger near-real-time updates, and low-priority events are temporarily stored in a batch update queue, waiting for a preset time window to trigger batch updates. Near-real-time updates are an update mode that falls between real-time updates and batch updates.

7. The method for establishing a spatiotemporal knowledge graph as described in claim 1, characterized in that, The method utilizes a spatiotemporal graph neural network on the spatiotemporal knowledge graph for link prediction and event deduction, combined with reinforcement learning for decision optimization, to output spatiotemporal reasoning results and decision schemes. These are then provided to application systems via an API interface, including: The spatiotemporal knowledge graph is input into a spatiotemporal graph neural network, which includes a spatial graph convolutional layer and a time series processing layer. The spatial graph convolutional layer aggregates spatial features, and the time series processing layer uses gated recurrent units to process the time series features of entities, predicting the spatial diffusion range of an event at a future time point and outputting the spatiotemporal evolution path of the event. Based on the node embedding vectors and relation embedding vectors generated during the spatiotemporal graph neural network processing, the energy value of the link is calculated, and the link prediction result is obtained: Define the state space as the current state of the spatiotemporal knowledge graph, the action space as the possible decision schemes, and the reward function as the degree of goal achievement after executing the action; The agent is trained using the DDPG algorithm. The actor network outputs deterministic actions based on the current state, and the critic network evaluates the value of the actions. The link prediction results and spatiotemporal evolution path are input into the trained agent to output the optimal decision scheme; The system integrates link prediction results, spatiotemporal evolution paths, and optimal decision-making schemes, pushes them to the application system via API, and records API call logs.

8. A spatiotemporal knowledge graph building device, characterized in that, The spatiotemporal knowledge graph building device includes: The spatiotemporal data acquisition and standardization module is used to acquire multi-source spatiotemporal data from sensor networks, geographic information systems, Internet of Things devices and text reports, and to preprocess the multi-source spatiotemporal data, including timestamp normalization, spatial coordinate unification and outlier filtering. The large-scale spatiotemporal semantic understanding and entity relationship extraction module is used to input pre-processed multi-source spatiotemporal data into a pre-trained large language model with spatiotemporal awareness capabilities, identify spatiotemporal entities, extract spatiotemporal relationships between entities, and output entities, relationship triples and their spatiotemporal trajectory fragments with spatiotemporal confidence. The spatiotemporal entity fusion and knowledge integration module is used to perform knowledge fusion on the output results using a spatiotemporal multimodal graph attention network. By calculating the spatial proximity, temporal series similarity and semantic similarity between entities, the attention weight of message passing between nodes in the graph is dynamically adjusted to form a spatiotemporal knowledge graph. The dynamic update and spatiotemporal evolution module is used to update the spatiotemporal knowledge graph in real time and record the spatiotemporal trajectory of entities based on an event-driven incremental update algorithm and a spatiotemporal trajectory tracking algorithm when new data streams enter. The spatiotemporal reasoning and application interface module is used to perform link prediction and event inference on the spatiotemporal knowledge graph using spatiotemporal graph neural networks, combined with reinforcement learning decision optimization, to output spatiotemporal reasoning results and decision schemes, and provide services to application systems through API interfaces.

9. A spatiotemporal knowledge graph building device, characterized in that, The spatiotemporal knowledge graph building device includes a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the spatiotemporal knowledge graph building device to perform the various steps of the spatiotemporal knowledge graph building method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the spatiotemporal knowledge graph establishment method as described in any one of claims 1-7.

Images