Retrieval method and device based on spatio-temporal knowledge graph attention network

Abstract

The invention discloses a retrieval method and device based on a space-time mapping knowledge domain attention network, and belongs to the technical field of mapping knowledge domains. The method comprises the following steps: acquiring a retrieval statement, extracting entities and entity relationships in the retrieval statement, distributing timestamps and space coordinates for the entities and the entity relationships, and constructing a space-time knowledge graph in combination with attribute information between the entities and the entity relationships; constructing an attention network model based on an encoder-decoder framework, and replacing attention mechanisms in an encoder and a decoder with an integrated attention mechanism; the time stamps, the space coordinates and the attribute information of the entities and the entity relations in the space-time knowledge graph are input into an encoder to generate high-dimensional feature vectors, the decoder decodes and outputs the high-dimensional feature vectors, and a retrieval result is obtained. According to the method, through ontology mapping of the space-time knowledge graph and knowledge migration of the integrated attention mechanism, space-time knowledge in one field can be applied to other fields, and the generalization ability of retrieval is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and more specifically, to a retrieval method and device based on a spatiotemporal knowledge graph attention network. Background Art

[0002] In a school environment, resource allocation and project management is a complex task, especially when it comes to interdisciplinary collaboration and resource sharing. Traditional retrieval systems often rely on keyword matching and static database queries, which have some shortcomings in dealing with dynamically changing information and complex relationship networks.

[0003] In schools, the relationships between projects, faculty, and colleges are multidimensional and dynamic. Traditional search methods, such as keyword-based searches, often fail to capture these complex relationships and their temporal variations. For example, a project may require the participation of experts from different fields at different stages, or a college may prioritize and invest resources in a specific project during a specific time period. If a search system fails to recognize these dynamic changes, it can lead to inaccurate information and wasted resources.

[0004] In school project search, spatiotemporal information can help the system understand the project's stage of progress, teacher and college availability, and changes in professional fields. For example, a project might be initiated at the beginning of the semester and require support from a specific college; at the end of the semester, resources from another college might be needed to complete the project. By integrating spatiotemporal information, the search system can more accurately match project needs with available resources.

[0005] In existing technologies, the attention mechanism is a computational model that mimics the selective focusing of human attention. It can help the system focus on the most relevant parts when processing large amounts of information. In school project retrieval, the attention mechanism can be used to identify the closest connections between projects and teachers and colleges. For example, the system can automatically focus on teachers and colleges that are most relevant to the project topic and have the most frequent historical collaborations, thereby improving retrieval efficiency and accuracy. However, when applied to knowledge graphs, existing attention mechanisms are unable to integrate spatiotemporal information, making it difficult to identify dynamic changes in the knowledge graph, resulting in low retrieval efficiency and accuracy. Summary of the Invention

[0006] 1. Technical problems to be solved In order to solve the above technical problems, the present invention provides a retrieval method and device based on a spatiotemporal knowledge graph attention network, which combines the spatiotemporal knowledge graph and the integrated attention mechanism to apply the spatiotemporal knowledge of one field to other fields, thereby improving the generalization ability of retrieval and the cross-domain retrieval effect.

[0007] 2. Technical solution The purpose of the present invention is achieved through the following technical solutions.

[0008] A retrieval method based on a spatiotemporal knowledge graph attention network includes the following steps: Obtain search statements, extract entities and entity relationships in the search statements, assign timestamps and spatial coordinates to the entities and entity relationships, and construct a spatiotemporal knowledge graph by combining the attribute information between the entities and entity relationships; An attention network model is constructed based on the encoder-decoder framework, where the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism. The timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph are input into the encoder to generate a high-dimensional feature vector. The decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval results.

[0009] As a further improvement of the present invention, the encoder of the attention network model includes a word embedding layer and an encoder layer; the encoder layer is composed of an integrated attention mechanism and a feedforward neural network.

[0010] As a further improvement of the present invention, the word embedding layer is expressed as:

[0011] Among them, X represents the word embedding layer vector, T represents a natural number, and x T Represents the embedding vector of the T-th word.

[0012] As a further improvement of the present invention, the integrated attention mechanism is expressed as:

[0013] Among them, CombinedAttention represents the integrated attention mechanism, Q represents the query vector, K represents the key vector, V represents the value vector, P represents the position encoding vector, g represents the gating function, Concat represents the connection operation, i, j and k all represent natural numbers, head i represents the output of the i-th head, R represents the learnable parameters, represents the output weight matrix, n represents the total number of key vectors, exp represents the exponential function, sim represents the dot product similarity function, represents the weight matrix of the i-th head used to transform the query vector, represents the j-th key vector, represents the weight matrix for transforming the key vector of the i-th head, represents the position encoding vector of the jth element, represents the sigmoid activation function, represents the kth key vector, represents the position encoding vector of the kth element, represents the j-th value vector, Represents the weight matrix used to transform the value vector of the i-th head.

[0014] As a further improvement of the present invention, the feedforward neural network is expressed as:

[0015] Among them, Z represents the input of the feedforward neural network, F(Z) represents the output of the feedforward neural network, W1 and W2 represent different weight matrices, and b1 and b2 represent different bias terms.

[0016] As a further improvement of the present invention, the decoder includes a decoder layer and an output layer; the decoder layer includes an integrated attention mechanism, a feedforward neural network and an encoder-decoder attention mechanism.

[0017] As a further improvement of the present invention, the weight calculation formula of the encoder-decoder attention mechanism is:

[0018] Among them, i and j both represent natural numbers. represents the attention weight of the jth position of the encoder output when the decoder generates the i-th output, represents the similarity score between the i-th output of the decoder and the j-th output of the encoder, k represents the position index in the encoder output sequence, T represents the total length of the encoder output sequence, represents the similarity score between the i-th output of the decoder and the k-th output of the encoder.

[0019] As a further improvement of the present invention, the output layer is expressed as:

[0020] Among them, O represents the original score of the output layer, Softmax represents the conversion of the output into a probability distribution, Y represents the output of the previous layer, W3 represents the weight of the output layer, and b3 represents the bias of the output layer.

[0021] As a further improvement of the present invention, a knowledge graph ontology is constructed through entities and entity relationships, timestamps and spatial coordinates are assigned to the knowledge graph ontology, and attribute information between entities and entity relationships is combined to construct a spatiotemporal knowledge graph.

[0022] A retrieval device based on a spatiotemporal knowledge graph attention network, comprising: The spatiotemporal knowledge graph construction module obtains search statements, extracts entities and entity relationships in the search statements, assigns timestamps and spatial coordinates to the entities and entity relationships, and combines the attribute information between the entities and entity relationships to construct a spatiotemporal knowledge graph; The attention network model building module builds an attention network model based on the encoder-decoder framework; in which the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism; The retrieval module inputs the timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph into the encoder to generate a high-dimensional feature vector. The decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval results.

[0023] 3. Beneficial effects Compared with the prior art, the advantages of the present invention are: (1) The present invention provides a retrieval method and device based on the attention network of the spatiotemporal knowledge graph. By constructing a spatiotemporal knowledge graph, the retrieval method and device realize multimodal spatiotemporal knowledge fusion. It not only integrates traditional structured spatiotemporal data, but also integrates multimodal data such as text, images, and videos, enriching the content and expression of the spatiotemporal knowledge graph. At the same time, dynamic edges are introduced through dynamic spatiotemporal relationship modeling to represent the relationship between entities changing over time, so that the spatiotemporal knowledge graph can accurately reflect the dynamic changes in the real world. Finally, the multi-level spatiotemporal knowledge structure constructed forms a coarse-to-fine knowledge hierarchy from macroscopic geographical areas and time spans to microscopic locations and the moment of event occurrence, facilitating efficient retrieval and reasoning in different scenarios.

[0024] (2) The present invention provides a retrieval method and device based on a spatiotemporal knowledge graph attention network. Based on the optimization of the retrieval strategy of spatiotemporal semantics, a retrieval algorithm for spatiotemporal knowledge graphs is proposed. An attention network model is constructed based on the encoder-decoder framework. The attention mechanism in the encoder is replaced with an integrated attention mechanism. Combined with streaming data processing technology, the real-time update and dynamic retrieval of the spatiotemporal knowledge graph are achieved, ensuring that the retrieval results can reflect the latest spatiotemporal information. Through the ontology mapping of the spatiotemporal knowledge graph and the knowledge transfer of the integrated attention mechanism, the spatiotemporal knowledge of one field is applied to other fields, improving the generalization ability of the retrieval and the cross-domain retrieval effect. The method is suitable for scenarios such as real-time monitoring and emergency response, and has strong practicality and wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a retrieval method according to an embodiment of the present invention; Figure 2 Schematic diagram of the attention network model structure constructed based on the encoder-decoder framework in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Example like Figure 1 As shown, a retrieval method based on the spatiotemporal knowledge graph attention network provided in this embodiment includes the following steps: obtaining a retrieval statement, extracting entities and entity relationships in the retrieval statement, assigning timestamps and spatial coordinates to the entities and entity relationships, and combining the attribute information between the entities and entity relationships to construct a spatiotemporal knowledge graph; constructing an attention network model based on the encoder-decoder framework, wherein the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism; inputting the timestamps, spatial coordinates and attribute information of the entities and entity relationships in the spatiotemporal knowledge graph into the encoder to generate a high-dimensional feature vector, and the decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval result.

[0028] Knowledge graphs are a knowledge representation technology that has evolved from traditional knowledge engineering and the development of the semantic web. They aim to describe concepts, entities, events, and their relationships in the objective world. A knowledge graph can also be viewed as a large graph, where nodes represent entities or concepts, and edges consist of attributes or relationships. Knowledge graphs have become a general term for various large-scale knowledge bases and have gradually permeated various fields. For example, in a school setting, resource allocation and project management are complex tasks, especially when it comes to interdisciplinary collaboration and resource sharing. Traditional search systems often rely on keyword matching and static database queries, which have limitations when dealing with dynamically changing information and complex relational networks. Therefore, there is a need to further construct spatiotemporal knowledge graphs that assign timestamps and spatial coordinates to entities such as teachers, colleges, and projects within a school, as well as the relationships between them. This, combined with attribute information between entities and their relationships, ensures efficient search services and accurate results for school resource allocation and project management. Conventional search methods often rely on keyword matching, which can be inaccurate when dealing with complex relationships and dynamically changing information. The retrieval method that combines spatiotemporal information can provide more accurate matching results by considering the actual relationships between projects, teachers, and colleges and the changes in the time dimension.

[0029] Specifically in this embodiment, a search statement is obtained, entities and entity relationships in the search statement are extracted, timestamps and spatial coordinates are assigned to entities and entity relationships, and the attribute information between entities and entity relationships is combined to construct a spatiotemporal knowledge graph. First, an ontology model is designed. The ontology model is the basis of the spatiotemporal knowledge graph and is used to define entity categories, attribute information, and entity relationships. In the field of education, entities include teachers, students, colleges, projects, pictures, and videos, while entity relationships include teacher-student relationships, project collaboration, etc. When designing an ontology model, it is necessary to determine the entity and entity relationship types, the classification level, and assign timestamps and spatial coordinates to each entity and entity relationship.

[0030] In this embodiment, entities in the field of education are classified into several categories, such as people, projects, equipment, and places, and timestamps and spatial coordinates are added to each entity category to record the evolution of entities and entity relationships. Specifically, for person entities, information such as date of birth, place of birth, and university of graduation is added; for project entities, different entities are used to represent the same project at different stages, and the start and end times are recorded; for equipment entities, the specific characteristics of the equipment are recorded; and for place entities, longitude and latitude are added to record geographic locations. Furthermore, relationships between entities, i.e., entity relationships, are defined, and attribute information such as time periods are added to entity relationships to represent relationships that exist within a specific time period, such as belonging to, located in, serving at, working at, etc.

[0031] Therefore, in this embodiment, data is obtained from different sources and converted into a graph format. Structured data is directly converted from formats such as JSON in the database, some structured data is extracted from web pages through crawlers, and open data uses natural language processing (NLP) technology to parse relationships from text. Furthermore, data from different data sources are processed in a unified manner, including entity linking, knowledge merging, and knowledge processing, to optimize the quantity and quality of knowledge. It should be noted that in this embodiment, entity linking is used to merge synonymous entities, knowledge merging is used to unify data from multiple databases, and knowledge processing is used to optimize the quantity and quality of knowledge. Finally, existing graph database tools, such as OrientDB, are used to store spatiotemporal data, and the spatiotemporal knowledge graph of the target field is constructed through spatiotemporal data.

[0032] By constructing a spatiotemporal knowledge graph, multimodal spatiotemporal knowledge fusion can be achieved. This not only integrates traditional structured spatiotemporal data, but also integrates multimodal data such as text, images, and videos, enriching the content and representation of the spatiotemporal knowledge graph. At the same time, through dynamic spatiotemporal relationship modeling, dynamic edges are introduced to represent the relationship between entities that change over time. For example, the process of "someone switching from Company A to Company B" enables the spatiotemporal knowledge graph to accurately reflect the dynamic changes in the real world. In addition, hierarchical spatiotemporal knowledge organization can be achieved. By constructing a multi-level spatiotemporal knowledge structure, a knowledge hierarchy is formed from coarse to fine, from macroscopic geographical areas and time spans to microscopic locations and the moment of event occurrence, facilitating efficient retrieval and reasoning in different scenarios.

[0033] Furthermore, if Figure 2 As shown, an attention network model is constructed based on the encoder-decoder framework. In this embodiment, the attention mechanism in the encoder and decoder is replaced with an integrated attention mechanism. It should be noted that replacing the attention mechanism in the encoder and decoder with an integrated attention mechanism enables the attention network model to dynamically focus on the most relevant information when processing sequence data. For example, through the Squeeze operation in the existing Squeeze-and-Excitation Networks (SENet), the attention network model can globally aggregate features and then adaptively adjust the importance of features through the Excitation operation.

[0034] In this example, for each element in the input spatiotemporal knowledge graph, including timestamps, spatial coordinates, and attribute information (e.g., teacher or college attributes), an attention score is calculated for the current task (e.g., project retrieval). The attention score can be implemented as a parameterized function that takes the input element and the current context vector as input and outputs a score.

[0035] The goal of the encoder is to encode data such as the timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph into a high-dimensional feature vector. In the education field, this includes project descriptions, teacher profiles, and school information. The encoder consists of multiple layers, including word embedding layers and encoder layers. Each encoder layer includes an integrated attention mechanism and a feed-forward network (FFN).

[0036] In this embodiment, the word embedding layer is represented as:

[0037] Among them, X represents the word embedding layer vector, T represents a natural number, and x TRepresents the embedding vector of the T-th word.

[0038] The integrated attention mechanism is expressed as:

[0039] Among them, CombinedAttention represents the integrated attention mechanism, Q represents the query vector, K represents the key vector, V represents the value vector, P represents the position encoding vector, g represents the gating function, Concat represents the connection operation, i represents a natural number, which is used to traverse each head in the multi-head attention, j represents a natural number, which is used to traverse each element in the input sequence in order to calculate the attention weight of each element and the final weighted value vector, k represents a natural number, which is used to traverse each element in the sequence in order to calculate the attention weight of each element, head i represents the output of the i-th head, R represents the learnable parameters, represents the output weight matrix, n represents the total number of key vectors, exp represents the exponential function, sim represents the dot product similarity function, represents the weight matrix of the i-th head used to transform the query vector, represents the j-th key vector, represents the weight matrix for transforming the key vector of the i-th head, represents the position encoding vector of the jth element, Represents the sigmoid activation function, which is used to convert the output of the gate function into a value between 0 and 1 to indicate the degree of attention. represents the kth key vector, represents the position encoding vector of the kth element, represents the j-th value vector, Represents the weight matrix used to transform the value vector of the i-th head.

[0040] The feedforward neural network is represented as:

[0041] Among them, Z represents the input of the feedforward neural network, F(Z) represents the output of the feedforward neural network, W1 and W2 represent different weight matrices, and b1 and b2 represent different bias terms.

[0042] By integrating the attention mechanism, the characteristics of multi-head attention, positional attention, and gated attention can be combined. In this embodiment, multi-head attention uses parallel computation of multiple heads, allowing the attention network model to simultaneously focus on different parts of the input. Positional attention uses the position code P, allowing the attention network model to consider the position information of elements in the sequence. Gated attention uses the gating function g, allowing the attention network model to control the degree of attention each head pays to different parts. The integrated attention mechanism can be used for complex tasks that consider multiple aspects of spatiotemporal information, multimodal information processing, and long sequence dependency analysis.

[0043] In this embodiment, the decoder decodes the encoder's output feature vector into understandable output, such as search results or recommended content. The decoder includes a decoder layer and an output layer. The decoder layer includes an integrated attention mechanism, a feedforward neural network, and an encoder-decoder attention mechanism.

[0044] The encoder-decoder attention mechanism allows the decoder to focus on specific parts of the encoder. In this embodiment, the weight calculation formula of the encoder-decoder attention mechanism is:

[0045] Among them, i and j both represent natural numbers. represents the attention weight of the jth position of the encoder output when the decoder generates the i-th output, represents the similarity score between the i-th output of the decoder and the j-th output of the encoder, k represents the position index in the encoder output sequence, T represents the total length of the encoder output sequence, represents the similarity score between the i-th output of the decoder and the k-th output of the encoder.

[0046] The weighted output is expressed as:

[0047] Among them, Y represents the context vector representing the decoder, represents the vector representing the j-th position of the encoder output.

[0048] For the output layer of the decoder, the attention scores are normalized using the softmax function so that the sum of all scores is 1, so that each score can be interpreted as a probability. In this embodiment, the output layer is represented as:

[0049] Among them, O represents the original score of the output layer, Softmax represents the conversion of the output into a probability distribution, Y represents the output of the previous layer, usually the context vector of the decoder, which contains the information output by the encoder, W3 represents the weight of the output layer, and b3 represents the bias of the output layer.

[0050] Therefore, according to the normalized attention score, the input sequence is weighted and summed to obtain the context vector, which will be used for subsequent retrieval or classification tasks.

[0051] Furthermore, the integrated attention mechanism is used to enhance the attention network model's ability to perceive spatiotemporal features based on the fusion of spatiotemporal features. For example, by considering the spatiotemporal features of the Geographic Knowledge Graph (GeoKG), a semantic network capable of describing geographic concepts, entities, and their relationships is constructed.

[0052] It should be noted that in this embodiment, a loss function is defined for training the attention network model, such as cross-entropy loss, and the model parameters are optimized using a backpropagation algorithm. During training, the attention network model learns how to assign attention weights to minimize the loss function.

[0053] In this embodiment, a classification problem is set. represents the true label, represents the predicted label of the attention network model, then the cross entropy loss function is defined as:

[0054] Where C represents the number of categories, represents the true label encoding, Represents the probability distribution predicted by the attention network model.

[0055] In this embodiment, the backpropagation algorithm is used to calculate the gradient of the loss function with respect to the parameters of the attention network model. Assuming that θ represents the parameters of the attention network model, the steps of the backpropagation algorithm are as follows: Forward propagation: computing the model’s output predictions ; Calculating loss: using loss function Calculate losses; Calculate the gradient: Use the chain rule to calculate ; Update parameters: Update the parameters θ using the gradient descent algorithm.

[0056] In this embodiment, during the training process, the attention network model learns how to allocate attention weights to minimize the loss function. The steps are as follows: Calculate attention weight: Use the integrated attention mechanism to calculate the attention weight α ij ; Weighted summation: Use the attention weights to perform weighted summation on the value vector V to obtain the output Z; Feedforward neural network: pass the output Z through the feedforward neural network to get the final prediction .

[0057] In this embodiment, the gradient of the loss function with respect to the parameters of the attention network model is calculated through the back propagation algorithm, and the parameters are updated using the gradient descent algorithm. In this embodiment, the formula of the gradient descent algorithm is:

[0058] Among them, θ new represents the updated parameters, θ old Indicates the parameters before updating, Represents the learning rate.

[0059] Finally, the elements of the spatiotemporal knowledge graph are input into the encoder to generate high-dimensional feature vectors. The decoder decodes the high-dimensional feature vectors and outputs the retrieval results. Specifically, based on the input question or task, one or more queries are generated that represent the user's needs. Retrieval is performed, using a retrieval system (e.g., based on TF-IDF, BM25, vector similarity, etc.) to find the most relevant documents or information fragments in the spatiotemporal knowledge graph. Document ranking ranks the retrieved documents based on relevance, selecting the most relevant documents as input for the generation phase. The retrieved documents or information fragments are used as contextual information and fed into the attention network model described above, along with the original question. Key features, namely high-dimensional feature vectors, are extracted from the retrieved documents. These key features guide the generation of the output of the attention network model. The attention network model described above, combined with the retrieved contextual information, generates an answer or completes the task. The generated answers undergo post-processing, including grammar checking and fact verification, to ensure accuracy and reliability. Finally, the attention network model is fine-tuned based on user feedback or human evaluation to improve retrieval efficiency.

[0060] Therefore, in this embodiment, the timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph are input into the encoder. The input text information is first converted into an embedding vector through the word embedding layer, so that each word has a corresponding vector representation, and a position code is set for each word. The resulting vector is then sent to the encoder's integrated attention mechanism for processing, so that the attention network model can pay attention to different parts of the input vector. The output of the integrated attention mechanism is then further nonlinearly transformed through a feedforward neural network. The output vector after nonlinear transformation enters the decoder. The output vector after nonlinear transformation passes through the encoder-decoder attention mechanism to obtain a decoding vector. The decoding vector is then processed through the integrated attention mechanism to obtain an output vector. The feedforward neural network performs nonlinear transformation on the output vector. Finally, the output vector after nonlinear transformation is mapped to the vocabulary through the output layer and reconverted into text information to obtain the retrieval result.

[0061] In this embodiment, the retrieval strategy is optimized based on spatiotemporal semantics, and the retrieval algorithm for the spatiotemporal knowledge graph uses spatiotemporal semantic information to sort and filter the retrieval results. For example, based on the chronological order of events and the proximity of geographical locations, knowledge with higher relevance is recommended first. In addition, real-time retrieval and dynamic updates can be achieved. Combined with streaming data processing technology, real-time updates and dynamic retrieval of the spatiotemporal knowledge graph can be achieved to ensure that the retrieval results can reflect the latest spatiotemporal information, which is suitable for scenarios such as real-time monitoring and emergency response. Finally, cross-domain knowledge transfer and retrieval enhancement can also be achieved. Through the ontology mapping of the spatiotemporal knowledge graph and the knowledge transfer mechanism of the attention network model, the spatiotemporal knowledge of one field can be applied to other fields, thereby improving the generalization ability of retrieval and the cross-domain retrieval effect.

[0062] Since traditional retrieval methods are often unable to effectively handle temporal dynamics, the retrieval method of this embodiment combined with spatiotemporal information can track the changes of entities over time. For example, it can identify the projects in which a specific teacher is involved at different time points, or the research focus of the college in different years. The integrated attention mechanism can help the attention network model focus on the most critical information and reduce the interference of irrelevant information, thereby improving retrieval efficiency. When processing large amounts of data, this method can quickly locate the most relevant information and significantly reduce retrieval time compared to traditional methods. The retrieval module that adapts to open domains and continuous learning enhancement technology provides an interface for introducing external knowledge, allowing the system to adapt to open domains and continuous learning scenarios, and can achieve efficient retrieval of research projects, colleges, and teachers within the school, thereby improving retrieval efficiency.

[0063] This embodiment also provides a retrieval device based on a spatiotemporal knowledge graph attention network, comprising a spatiotemporal knowledge graph construction module, an attention network model construction module, and a retrieval module. The spatiotemporal knowledge graph construction module is used to obtain a search statement, extract entities and entity relationships in the search statement, assign timestamps and spatial coordinates to the entities and entity relationships, and construct a spatiotemporal knowledge graph. The attention network model construction module constructs an attention network model based on an encoder-decoder framework, wherein the attention mechanism in the encoder and decoder is replaced with an integrated attention mechanism. The retrieval module is used to input elements in the spatiotemporal knowledge graph into the encoder to generate a high-dimensional feature vector, and the decoder decodes and outputs the high-dimensional feature vector to obtain a retrieval result. The retrieval device based on a spatiotemporal knowledge graph attention network provided in this embodiment can implement any of the retrieval methods based on the spatiotemporal knowledge graph attention network, and the specific working process of a retrieval device based on a spatiotemporal knowledge graph attention network can refer to the corresponding process in the embodiment of the retrieval method based on the spatiotemporal knowledge graph attention network. The method and device provided in this embodiment can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of a module is merely a logical functional division, and actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, the connections or communication links shown or discussed may be indirect couplings or communication links via interfaces, devices, or units, or may be electrical, mechanical, or other forms of connection.

[0064] This embodiment also provides a computer device. A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the retrieval method based on the spatiotemporal knowledge graph attention network when executing the computer program.

[0065] This embodiment also provides a computer-readable storage medium. A computer-readable storage medium stores a computer program, which, when executed by a processor, executes a retrieval method based on a spatiotemporal knowledge graph attention network described in this embodiment. The computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0066] The above schematically describes the invention and its implementation methods. This description is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it and designs a structural method and embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the scope of protection of the present invention. In addition, the word "including" does not exclude other elements or steps, and the word "one" before an element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. Words such as first and second are used to indicate names and do not indicate any specific order.

Claims

1. A retrieval method based on a spatiotemporal knowledge graph attention network, comprising the following steps: Obtain search statements, extract entities and entity relationships in the search statements, assign timestamps and spatial coordinates to entities and entity relationships, and construct a spatiotemporal knowledge graph by combining attribute information between entities and entity relationships; An attention network model is constructed based on the encoder-decoder framework, wherein the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism; The timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph are input into the encoder to generate a high-dimensional feature vector. The decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval results.

2. According to claim 1, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The encoder of the attention network model includes a word embedding layer and an encoder layer; the encoder layer is composed of an integrated attention mechanism and a feedforward neural network.

3. According to claim 2, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The word embedding layer is expressed as: Among them, X represents the word embedding layer vector, T represents a natural number, and x T Represents the embedding vector of the Tth word.

4. According to claim 3, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The integrated attention mechanism is expressed as: Among them, CombinedAttention represents the integrated attention mechanism, Q represents the query vector, K represents the key vector, V represents the value vector, P represents the position encoding vector, g represents the gating function, Concat represents the connection operation, i, j and k all represent natural numbers, head i represents the output of the i-th head, R represents the learnable parameters, represents the output weight matrix, n represents the total number of key vectors, exp represents the exponential function, sim represents the dot product similarity function, represents the weight matrix of the i-th head used to transform the query vector, represents the j-th key vector, represents the weight matrix used to transform the key vector of the i-th head, represents the position encoding vector of the jth element, represents the sigmoid activation function, represents the kth key vector, represents the positional encoding vector of the kth element, represents the jth value vector, Represents the weight matrix used to transform the value vector of the i-th head.

5. According to claim 4, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The feedforward neural network is expressed as: Among them, Z represents the input of the feedforward neural network, F(Z) represents the output of the feedforward neural network, W1 and W2 represent different weight matrices, and b1 and b2 represent different bias items.

6. A retrieval method based on spatiotemporal knowledge graph attention network according to claim 5, characterized in that: The decoder includes a decoder layer and an output layer; the decoder layer includes an integrated attention mechanism, a feedforward neural network and an encoder-decoder attention mechanism.

7. A retrieval method based on spatiotemporal knowledge graph attention network according to claim 6, characterized in that: The weight calculation formula of the encoder-decoder attention mechanism is: Among them, i and j both represent natural numbers. represents the attention weight of the jth position of the encoder output when the decoder generates the i-th output, represents the similarity score between the i-th output of the decoder and the j-th output of the encoder, k represents the position index in the encoder output sequence, T represents the total length of the encoder output sequence, represents the similarity score between the i-th output of the decoder and the k-th output of the encoder.

8. A retrieval method based on spatiotemporal knowledge graph attention network according to claim 7, characterized in that: The output layer is expressed as: Among them, O represents the original score of the output layer, Softmax represents the conversion of the output into a probability distribution, Y represents the output of the previous layer, W3 represents the weight of the output layer, and b3 represents the bias of the output layer.

9. The retrieval method based on spatiotemporal knowledge graph attention network according to claim 1, characterized in that: The knowledge graph ontology is constructed through entities and entity relationships, timestamps and spatial coordinates are assigned to the knowledge graph ontology, and the attribute information between entities and entity relationships is combined to construct a spatiotemporal knowledge graph.

10. A retrieval device based on a spatiotemporal knowledge graph attention network, characterized in that: include: The spatiotemporal knowledge graph construction module obtains the search statement, extracts the entities and entity relationships in the search statement, assigns timestamps and spatial coordinates to the entities and entity relationships, and combines the attribute information between the entities and entity relationships to construct the spatiotemporal knowledge graph; The attention network model building module builds an attention network model based on the encoder-decoder framework, where the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism; The retrieval module inputs the timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph into the encoder to generate a high-dimensional feature vector. The decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval results.

Images