A retrieval method and device based on spatiotemporal knowledge graph attention network

Abstract

The present invention discloses a retrieval method and device based on a spatiotemporal knowledge graph attention network, belonging to the field of knowledge graph technology. The method comprises: obtaining a search statement, extracting entities and entity relationships in the search statement, assigning timestamps and spatial coordinates to entities and entity relationships, combining the attribute information between entities and entity relationships, and constructing a spatiotemporal knowledge graph; constructing an attention network model based on an encoder-decoder framework, replacing the attention mechanism in the encoder and decoder with an integrated attention mechanism; inputting the timestamps, spatial coordinates and attribute information of entities and entity relationships in the spatiotemporal knowledge graph into the encoder to generate a high-dimensional feature vector, and the decoder decodes and outputs the high-dimensional feature vector to obtain a search result. The present invention can apply the spatiotemporal knowledge of one field to other fields through the ontology mapping of the spatiotemporal knowledge graph and the knowledge transfer of the integrated attention mechanism, and significantly improve the generalization ability of the retrieval.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and more specifically, to a retrieval method and device based on a spatiotemporal knowledge graph attention network. Background Art

[0002] In a school environment, resource allocation and project management is a complex task, especially when it comes to interdisciplinary collaboration and resource sharing. Traditional retrieval systems often rely on keyword matching and static database queries, which have some shortcomings in dealing with dynamically changing information and complex relationship networks.

[0003] In schools, the relationships among projects, teachers, and colleges are multi-dimensional and dynamically changing. Traditional retrieval methods, such as keyword-based searches, often fail to capture these complex relationships and temporal changes. For example, a project may require the participation of experts in different fields at different stages, or a college may pay special attention to and invest resources in a specific project during a specific period of time. If the retrieval system cannot identify these dynamic changes, it may lead to inaccurate information and waste of resources.

[0004] In school project retrieval, spatiotemporal information can help the system understand the progress stage of the project, the availability of teachers and colleges, and changes in professional fields. For example, a project may be launched at the beginning of the semester and require support from a specific college; at the end of the semester, resources from another college may be needed to complete the project. By integrating spatiotemporal information, the retrieval system can more accurately match project requirements and available resources.

[0005] In the prior art, the attention mechanism is a computational model that mimics the selective focus of human attention. It can help the system focus on the most relevant parts when processing a large amount of information. In school project retrieval, the attention mechanism can be used to identify the closest connection between the project and the teacher and college. For example, the system can automatically focus on the teachers and colleges that are most relevant to the project topic and have the most frequent historical collaborations, thereby improving the efficiency and accuracy of retrieval. However, when the existing attention mechanism is applied to the knowledge graph, it is unable to integrate spatiotemporal information, making it difficult to identify dynamic changes in the knowledge graph, resulting in low retrieval efficiency and retrieval accuracy. Summary of the invention

[0006] 1. Technical issues to be solved

[0007] In order to solve the above technical problems, the present invention provides a retrieval method and device based on a spatiotemporal knowledge graph attention network, which combines the spatiotemporal knowledge graph and the integrated attention mechanism to apply the spatiotemporal knowledge of one field to other fields, thereby improving the generalization ability of retrieval and the cross-domain retrieval effect.

[0008] 2. Technical solution

[0009] The purpose of the present invention is achieved through the following technical solutions.

[0010] A retrieval method based on a spatiotemporal knowledge graph attention network comprises the following steps:

[0011] Obtain search statements, extract entities and entity relationships in the search statements, assign timestamps and spatial coordinates to entities and entity relationships, and construct a spatiotemporal knowledge graph by combining attribute information between entities and entity relationships;

[0012] An attention network model is constructed based on the encoder-decoder framework, wherein the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism;

[0013] The timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph are input into the encoder to generate a high-dimensional feature vector. The decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval results.

[0014] As a further improvement of the present invention, the encoder of the attention network model includes a word embedding layer and an encoder layer; the encoder layer is composed of an integrated attention mechanism and a feedforward neural network.

[0015] As a further improvement of the present invention, the word embedding layer is expressed as:

[0016]

[0017] Among them, X represents the word embedding layer vector, T represents a natural number, and x T Represents the embedding vector of the Tth word.

[0018] As a further improvement of the present invention, the integrated attention mechanism is expressed as:

[0019]

[0020] Among them, CombinedAttention represents the integrated attention mechanism, Q represents the query vector, K represents the key vector, V represents the value vector, P represents the position encoding vector, g represents the gating function, Concat represents the connection operation, i, j and k all represent natural numbers, head i represents the output of the i-th head, R represents the learnable parameters, represents the output weight matrix, n represents the total number of key vectors, exp represents the exponential function, sim represents the dot product similarity function, represents the weight matrix of the i-th head used to transform the query vector, represents the j-th key vector, represents the weight matrix used to transform the key vector of the i-th head, represents the position encoding vector of the jth element, represents the sigmoid activation function, represents the kth key vector, represents the positional encoding vector of the kth element, represents the j-th value vector, Represents the weight matrix used to transform the value vector of the i-th head.

[0021] As a further improvement of the present invention, the feedforward neural network is expressed as:

[0022]

[0023] Where Z represents the input of the feedforward neural network, F(Z) represents the output of the feedforward neural network, and W 1 , W 2 represents different weight matrices, b 1 、b 2 Represents different bias terms.

[0024] As a further improvement of the present invention, the decoder includes a decoder layer and an output layer; the decoder layer includes an integrated attention mechanism, a feedforward neural network and an encoder-decoder attention mechanism.

[0025] As a further improvement of the present invention, the calculation formula of the weight of the encoder-decoder attention mechanism is:

[0026]

[0027] Among them, i and j both represent natural numbers. represents the attention weight of the jth position of the encoder output when the decoder generates the i-th output, represents the similarity score between the i-th output of the decoder and the j-th output of the encoder, k represents the position index in the encoder output sequence, T represents the total length of the encoder output sequence, represents the similarity score between the i-th output of the decoder and the k-th output of the encoder.

[0028] As a further improvement of the present invention, the output layer is expressed as:

[0029]

[0030] Among them, O represents the original score of the output layer, Softmax represents the conversion of the output into a probability distribution, Y represents the output of the previous layer, and W 3 represents the weight of the output layer, b 3 Represents the bias of the output layer.

[0031] As a further improvement of the present invention, a knowledge graph ontology is constructed through entities and entity relationships, timestamps and spatial coordinates are assigned to the knowledge graph ontology, and attribute information between entities and entity relationships is combined to construct a spatiotemporal knowledge graph.

[0032] A retrieval device based on a spatiotemporal knowledge graph attention network, comprising:

[0033] The spatiotemporal knowledge graph construction module obtains the search statement, extracts the entities and entity relationships in the search statement, assigns timestamps and spatial coordinates to the entities and entity relationships, and combines the attribute information between the entities and entity relationships to construct the spatiotemporal knowledge graph;

[0034] The attention network model building module builds an attention network model based on the encoder-decoder framework, where the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism;

[0035] The retrieval module inputs the timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph into the encoder to generate a high-dimensional feature vector. The decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval results.

[0036] 3. Beneficial effects

[0037] Compared with the prior art, the advantages of the present invention are:

[0038] (1) The present invention provides a retrieval method and device based on the attention network of the spatiotemporal knowledge graph, which realizes multimodal spatiotemporal knowledge fusion by constructing a spatiotemporal knowledge graph. It not only integrates traditional structured spatiotemporal data, but also integrates multimodal data such as text, images, and videos, enriching the content and expression of the spatiotemporal knowledge graph. At the same time, dynamic edges are introduced through dynamic spatiotemporal relationship modeling to represent the relationship between entities that change over time, so that the spatiotemporal knowledge graph can accurately reflect the dynamic changes in the real world. Finally, the multi-level spatiotemporal knowledge structure constructed forms a knowledge hierarchy from coarse to fine, from macroscopic geographical areas and time spans to microscopic locations and the moment when events occur, facilitating efficient retrieval and reasoning in different scenarios.

[0039] (2) The present invention provides a retrieval method and device based on the attention network of spatiotemporal knowledge graph, optimizes the retrieval strategy based on spatiotemporal semantics, proposes a retrieval algorithm for spatiotemporal knowledge graph, builds an attention network model based on the encoder-decoder framework, replaces the attention mechanism in the encoder with an integrated attention mechanism, and combines streaming data processing technology to achieve real-time update and dynamic retrieval of spatiotemporal knowledge graph, ensuring that the retrieval results can reflect the latest spatiotemporal information. Through the ontology mapping of the spatiotemporal knowledge graph and the knowledge transfer of the integrated attention mechanism, the spatiotemporal knowledge of one field is applied to other fields, improving the generalization ability of the retrieval and the cross-domain retrieval effect, which is suitable for scenarios such as real-time monitoring and emergency response, and has strong practicality and wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of a retrieval method according to an embodiment of the present invention;

[0041] Figure 2 Schematic diagram of the structure of the attention network model constructed based on the encoder-decoder framework according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0043] Example

[0044] like Figure 1 As shown, a retrieval method based on the attention network of the spatiotemporal knowledge graph provided for this implementation includes the following steps: obtaining a retrieval statement, extracting entities and entity relationships in the retrieval statement, assigning timestamps and spatial coordinates to the entities and entity relationships, and combining the attribute information between the entities and entity relationships to construct a spatiotemporal knowledge graph; constructing an attention network model based on an encoder-decoder framework, wherein the attention mechanism in the encoder and decoder is replaced with an integrated attention mechanism; the timestamps, spatial coordinates and attribute information of the entities and entity relationships in the spatiotemporal knowledge graph are input into the encoder to generate a high-dimensional feature vector, and the decoder decodes and outputs the high-dimensional feature vector to obtain a retrieval result.

[0045] Knowledge graph is a knowledge representation technology that has been bred and developed on the basis of traditional knowledge engineering and the development of semantic web. It aims to describe concepts, entities, events and their relationships in the objective world. Knowledge graph can also be regarded as a huge graph, in which nodes represent entities or concepts, and edges are composed of attributes or relationships. Knowledge graph has been used to refer to various large-scale knowledge bases and gradually penetrated into various fields. For example, in the school environment, resource allocation and project management are complex tasks, especially when it comes to interdisciplinary cooperation and resource sharing. Traditional retrieval systems often rely on keyword matching and static database queries. These methods have some shortcomings when dealing with dynamically changing information and complex relationship networks. Therefore, it is necessary to further construct spatiotemporal knowledge graphs, assign timestamps and spatial coordinates to entities such as teachers, colleges, projects and their entity relationships in the school, and combine the attribute information between entities and entity relationships to provide guarantees for efficient retrieval services and accurate retrieval results for school resource allocation and project management. In the prior art, traditional retrieval methods may rely on keyword matching, which may not be accurate enough when dealing with complex relationships and dynamically changing information. The retrieval method that combines spatiotemporal information can provide more accurate matching results by considering the actual relationship between projects, teachers, and colleges and the changes in the time dimension.

[0046] Specifically in this embodiment, a search statement is obtained, entities and entity relationships in the search statement are extracted, timestamps and spatial coordinates are assigned to entities and entity relationships, and the attribute information between entities and entity relationships is combined to construct a spatiotemporal knowledge graph. First, an ontology model is designed. The ontology model is the basis of the spatiotemporal knowledge graph and is used to define entity categories, attribute information, and entity relationships. In the field of education, entities include teachers, students, colleges, projects, pictures, and videos, while entity relationships include teacher-student relationships, project collaboration, etc. When designing an ontology model, it is necessary to determine the entity and entity relationship types, classification levels, and assign timestamps and spatial coordinates to each entity and entity relationship.

[0047] In this embodiment, entities in the field of education are classified into several categories, such as people, projects, equipment, and places, and timestamps and spatial coordinates are added to each entity category to record the changes in entities and entity relationships. Specifically, for person entities, information such as date of birth, place of birth, and graduate school is added; for project entities, different entities are used to represent the same project at different stages, and the start time and end time are recorded; for equipment entities, specific features of the equipment are recorded; for place entities, longitude and latitude are added to record geographic locations. Furthermore, the relationship between entities, i.e., entity relationships, is defined, and attribute information such as time periods are added to entity relationships to indicate relationships that exist within a specific time period, such as belonging to, located in, employed by, and worked for.

[0048] Therefore, in this embodiment, data is obtained from different sources and converted into a graph format. Structured data is directly converted from formats such as JSON in the database, some structured data is extracted from web pages through crawlers, and open data uses natural language processing (NLP) technology to parse relationships from text. Furthermore, data from different data sources are processed uniformly, including entity linking, knowledge merging, and knowledge processing to optimize the quantity and quality of knowledge. It should be noted that in this embodiment, entity linking is used to merge synonymous entities, knowledge merging is used to uniformly process data from multiple databases, and knowledge processing is used to optimize the quantity and quality of knowledge. Finally, existing graph database tools, such as OrientDB, are used to store spatiotemporal data, and spatiotemporal knowledge graphs of the target field are constructed through spatiotemporal data.

[0049] By constructing a spatiotemporal knowledge graph, multimodal spatiotemporal knowledge fusion can be achieved. It can not only integrate traditional structured spatiotemporal data, but also integrate multimodal data such as text, images, and videos to enrich the content and expression of the spatiotemporal knowledge graph. At the same time, through dynamic spatiotemporal relationship modeling, dynamic edges are introduced to represent the relationship between entities that change over time. For example, the process of "someone switching from Company A to Company B" enables the spatiotemporal knowledge graph to accurately reflect the dynamic changes in the real world. In addition, hierarchical spatiotemporal knowledge organization can also be achieved by constructing a multi-level spatiotemporal knowledge structure, from macroscopic geographical areas and time spans to microscopic locations and the moment when events occur, forming a knowledge hierarchy from coarse to fine, which is convenient for efficient retrieval and reasoning in different scenarios.

[0050] Furthermore, if Figure 2 As shown, an attention network model is constructed based on the encoder-decoder framework. In this embodiment, the attention mechanism in the encoder and decoder is replaced with an integrated attention mechanism. It should be noted that replacing the attention mechanism in the encoder and decoder with an integrated attention mechanism enables the attention network model to dynamically focus on the most relevant information when processing sequence data. For example, through the Squeeze operation in the existing Squeeze-and-Excitation Networks (SENet), the attention network model can globally aggregate features, and then adaptively adjust the importance of the features through the Excitation operation.

[0051] In this embodiment, for each element in the input spatiotemporal knowledge graph, including timestamp, spatial coordinates and attribute information (e.g., attributes of teachers and colleges), its attention score for the current task (e.g., project retrieval) is calculated. The attention score can be implemented by a parameterized function that takes the input element and the current context vector as input and outputs a score.

[0052] The goal of the encoder is to encode data such as timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph into a high-dimensional feature vector. In the field of education, it includes project descriptions, teacher information, college information, etc. The encoder consists of multiple layers, including word embedding layers and encoder layers. Each encoder layer contains an integrated attention mechanism and a feed-forward network (FFN).

[0053] In this embodiment, the word embedding layer is expressed as:

[0054]

[0055] Among them, X represents the word embedding layer vector, T represents a natural number, and x T Represents the embedding vector of the Tth word.

[0056] The integrated attention mechanism is expressed as:

[0057]

[0058] Among them, CombinedAttention represents the integrated attention mechanism, Q represents the query vector, K represents the key vector, V represents the value vector, P represents the position encoding vector, g represents the gating function, Concat represents the connection operation, i represents a natural number, which is used to traverse each head in the multi-head attention, j represents a natural number, which is used to traverse each element in the input sequence in order to calculate the attention weight of each element and the final weighted value vector, k represents a natural number, which is used to traverse each element in the sequence in order to calculate the attention weight of each element, head i represents the output of the i-th head, R represents the learnable parameters, represents the output weight matrix, n represents the total number of key vectors, exp represents the exponential function, sim represents the dot product similarity function, represents the weight matrix of the i-th head used to transform the query vector, represents the j-th key vector, represents the weight matrix used to transform the key vector of the i-th head, represents the position encoding vector of the jth element, Represents the sigmoid activation function, which is used to convert the output of the gating function into a value between 0 and 1 to indicate the degree of attention. represents the kth key vector, represents the positional encoding vector of the kth element, represents the j-th value vector, Represents the weight matrix used to transform the value vector of the i-th head.

[0059] The feedforward neural network is represented as:

[0060]

[0061] Where Z represents the input of the feedforward neural network, F(Z) represents the output of the feedforward neural network, and W 1 , W 2 represents different weight matrices, b 1 , b 2 Represents different bias terms.

[0062] By integrating the attention mechanism, the characteristics of multi-head attention, position attention and gated attention can be combined. In this embodiment, multi-head attention uses parallel calculations of multiple heads, and the attention network model can simultaneously focus on different parts of the input. Position attention uses position encoding P, and the attention network model can consider the position information of elements in the sequence. Gated attention uses the gating function g, and the attention network model can control the degree of attention of each head to different parts. The integrated attention mechanism can be used for complex tasks that consider multi-faceted spatiotemporal information, multimodal information processing, and long sequence dependency analysis.

[0063] In this embodiment, the decoder decodes the output feature vector of the encoder into an understandable output, such as a search result or recommended content. The decoder includes a decoder layer and an output layer. The decoder layer includes an integrated attention mechanism, a feedforward neural network, and an encoder-decoder attention mechanism.

[0064] The encoder-decoder attention mechanism allows the decoder to focus on specific parts of the encoder. In this embodiment, the weight calculation formula of the encoder-decoder attention mechanism is:

[0065]

[0066] Among them, i and j both represent natural numbers. represents the attention weight of the jth position of the encoder output when the decoder generates the i-th output, represents the similarity score between the i-th output of the decoder and the j-th output of the encoder, k represents the position index in the encoder output sequence, T represents the total length of the encoder output sequence, represents the similarity score between the i-th output of the decoder and the k-th output of the encoder.

[0067] The weight output is expressed as:

[0068]

[0069] Among them, Y represents the context vector representing the decoder, Denotes the vector representing the j-th position of the encoder output.

[0070] For the output layer of the decoder, the attention scores are normalized using the softmax function so that the sum of all scores is 1, so that each score can be interpreted as a probability. In this embodiment, the output layer is represented as:

[0071]

[0072] Among them, O represents the original score of the output layer, Softmax represents the conversion of the output into a probability distribution, Y represents the output of the previous layer, usually the context vector of the decoder, which contains the information of the encoder output, and W 3 represents the weight of the output layer, b 3 Represents the bias of the output layer.

[0073] Therefore, according to the normalized attention score, the input sequence is weighted and summed to obtain the context vector, which will be used for subsequent retrieval or classification tasks.

[0074] Furthermore, based on the fusion of spatiotemporal features, the integrated attention mechanism is used to enhance the perception of spatiotemporal features of the attention network model. For example, by considering the spatiotemporal features of the geographic knowledge graph (GeoKG), a semantic network that can describe geographic concepts, entities and their relationships is constructed.

[0075] It should be noted that, in this embodiment, a loss function is defined for training the attention network model, for example, cross-entropy loss, and the model parameters are optimized by the back propagation algorithm. During the training process, the attention network model learns how to allocate attention weights to minimize the loss function.

[0076] In this embodiment, a classification problem is set. represents the true label, represents the attention network model predicting the label, then the cross entropy loss function is defined as:

[0077]

[0078] Where C represents the number of categories, represents the true label encoding, Represents the probability distribution predicted by the attention network model.

[0079] In this embodiment, the back propagation algorithm is used to calculate the gradient of the loss function with respect to the parameters of the attention network model. Assuming that θ represents the parameters of the attention network model, the steps of the back propagation algorithm are as follows:

[0080] Forward propagation: computing the model’s output predictions ;

[0081] Calculating loss: Using loss function Calculate the loss;

[0082] Calculate the gradient: Use the chain rule to calculate ;

[0083] Update parameters: Update the parameters θ using the gradient descent algorithm.

[0084] In this embodiment, during the training process, the attention network model learns how to allocate attention weights to minimize the loss function. The steps are as follows:

[0085] Calculate attention weight: Use the integrated attention mechanism to calculate the attention weight α ij ;

[0086] Weighted summation: Use the attention weights to perform weighted summation on the value vector V to get the output Z;

[0087] Feedforward neural network: pass the output Z through the feedforward neural network to get the final prediction .

[0088] In this embodiment, the gradient of the loss function with respect to the parameters of the attention network model is calculated through the back propagation algorithm, and the parameters are updated using the gradient descent algorithm. In this embodiment, the formula of the gradient descent algorithm is:

[0089]

[0090] Among them, θ new represents the updated parameters, θ old Indicates the parameters before updating. Represents the learning rate.

[0091] Finally, the elements in the spatiotemporal knowledge graph are input into the encoder to generate a high-dimensional feature vector, and the decoder decodes the high-dimensional feature vector and outputs it to obtain the retrieval result. Specifically, one or more queries are generated based on the input question or task, which can represent the user's needs. Perform retrieval, using the retrieval system (for example, based on TF-IDF, BM25, vector similarity, etc.) to find the most relevant documents or information fragments in the spatiotemporal knowledge graph. Document sorting, sorting the retrieved documents according to relevance, and selecting the most relevant documents as input for the generation phase. The retrieved documents or information fragments are used as context information and input into the attention network model mentioned above together with the original question. Extract key features, i.e., high-dimensional feature vectors, from the retrieved documents. These key features will be used to guide the generation of the output of the attention network model. Use the attention network model mentioned above, combined with the retrieved context information, to generate answers or complete tasks. Post-process the generated answers, including grammar checking, fact verification, etc., to ensure the accuracy and reliability of the answers. Finally, fine-tune the attention network model based on user feedback or manual evaluation to improve retrieval efficiency.

[0092] Therefore, in this embodiment, the timestamps, spatial coordinates and attribute information of entities and entity relationships in the spatiotemporal knowledge graph are input into the encoder, and the input text information is first converted into an embedded vector through the word embedding layer, so that each word has a corresponding vector representation, and a position code is set for each word, and then the obtained vector is sent to the integrated attention mechanism of the encoder for processing, so that the attention network model can pay attention to different parts of the input vector, and then the output of the integrated attention mechanism is further nonlinearly transformed through the feedforward neural network. The output vector after nonlinear transformation enters the decoder, and the output vector after nonlinear transformation passes through the encoder-decoder attention mechanism to obtain a decoding vector, and the decoding vector is processed by the integrated attention mechanism to obtain an output vector, and the feedforward neural network performs a nonlinear transformation on the output vector, and finally the output vector after nonlinear transformation is mapped to the vocabulary through the output layer, and re-transformed into text information to obtain the retrieval result.

[0093] In this embodiment, the retrieval strategy is optimized based on spatiotemporal semantics, and the retrieval algorithm for the spatiotemporal knowledge graph uses spatiotemporal semantic information to sort and filter the retrieval results. For example, based on the chronological order of events and the proximity of geographical locations, knowledge with higher relevance is recommended first. In addition, real-time retrieval and dynamic updates can be realized. Combined with streaming data processing technology, real-time updates and dynamic retrieval of the spatiotemporal knowledge graph can be realized to ensure that the retrieval results can reflect the latest spatiotemporal information, which is suitable for scenarios such as real-time monitoring and emergency response. Finally, cross-domain knowledge transfer and retrieval enhancement can also be realized. Through the ontology mapping of the spatiotemporal knowledge graph and the knowledge transfer mechanism of the attention network model, the spatiotemporal knowledge of one field can be applied to other fields, thereby improving the generalization ability of retrieval and the cross-domain retrieval effect.

[0094] Since traditional retrieval methods are often unable to effectively handle time dynamics, the retrieval method of this embodiment combined with spatiotemporal information can track the changes of entities over time. For example, it can identify the projects in which a specific teacher is involved at different time points, or the research focus of the college in different years. The integrated attention mechanism can help the attention network model focus on the most critical information and reduce the interference of irrelevant information, thereby improving retrieval efficiency. When processing large amounts of data, this method can quickly locate the most relevant information, which can significantly reduce retrieval time compared to traditional methods. The retrieval module that adapts to open domains and continuous learning enhancement technology provides an interface for introducing external knowledge, so that the system can adapt to open domains and continuous learning scenarios, and can achieve efficient retrieval of research projects, colleges, and teachers within the school, thereby improving retrieval efficiency.

[0095] This embodiment also provides a retrieval device based on a spatiotemporal knowledge graph attention network, including a spatiotemporal knowledge graph construction module, an attention network model construction module and a retrieval module. The spatiotemporal knowledge graph construction module is used to obtain a search statement, extract entities and entity relationships in the search statement, assign timestamps and spatial coordinates to entities and entity relationships, and construct a spatiotemporal knowledge graph. The attention network model construction module constructs an attention network model based on an encoder-decoder framework, wherein the attention mechanism in the encoder and decoder is replaced with an integrated attention mechanism. The retrieval module is used to input the elements in the spatiotemporal knowledge graph into the encoder to generate a high-dimensional feature vector, and the decoder decodes and outputs the high-dimensional feature vector to obtain a retrieval result. A retrieval device based on a spatiotemporal knowledge graph attention network provided in this embodiment can implement any of the retrieval methods based on the spatiotemporal knowledge graph attention network, and the specific working process of a retrieval device based on a spatiotemporal knowledge graph attention network can refer to the corresponding process in the embodiment of the retrieval method based on the spatiotemporal knowledge graph attention network. The method and device provided in this embodiment can be implemented in other ways. For example, the device embodiments described above are only illustrative; for example, the division of a module is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual connection or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0096] This embodiment also provides a computer device. A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the retrieval method based on the spatiotemporal knowledge graph attention network when executing the computer program.

[0097] This embodiment also provides a computer-readable storage medium. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a retrieval method based on a spatiotemporal knowledge graph attention network described in this embodiment is executed. Among them, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0098] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited thereto, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all belong to the protection scope of the present invention. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate the name, and do not indicate any specific order.

Claims

1. A retrieval method based on a spatiotemporal knowledge graph attention network, comprising the following steps: Obtain search statements, extract entities and entity relationships in the search statements, assign timestamps and spatial coordinates to entities and entity relationships, and construct a spatiotemporal knowledge graph by combining attribute information between entities and entity relationships; An attention network model is constructed based on the encoder-decoder framework, wherein the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism; The timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph are input into the encoder to generate a high-dimensional feature vector. The decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval result. The integrated attention mechanism is expressed as: Among them, CombinedAttention represents the integrated attention mechanism, Q represents the query vector, K represents the key vector, V represents the value vector, P represents the position encoding vector, g represents the gating function, Concat represents the connection operation, i, j and k all represent natural numbers, head i represents the output of the i-th head, R represents the learnable parameters, represents the output weight matrix, n represents the total number of key vectors, exp represents the exponential function, sim represents the dot product similarity function, represents the weight matrix of the i-th head used to transform the query vector, represents the j-th key vector, represents the weight matrix used to transform the key vector of the i-th head, represents the position encoding vector of the jth element, represents the sigmoid activation function, represents the kth key vector, represents the positional encoding vector of the kth element, represents the j-th value vector, Represents the weight matrix used to transform the value vector of the i-th head.

2. According to claim 1, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The encoder of the attention network model includes a word embedding layer and an encoder layer; the encoder layer is composed of an integrated attention mechanism and a feedforward neural network.

3. According to claim 2, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The word embedding layer is expressed as: Among them, X represents the word embedding layer vector, T represents a natural number, and x T Represents the embedding vector of the Tth word.

4. According to claim 1, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The feedforward neural network is expressed as: Among them, Z represents the input of the feedforward neural network, F(Z) represents the output of the feedforward neural network, W1 and W2 represent different weight matrices, and b1 and b2 represent different bias items.

5. According to claim 4, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The decoder includes a decoder layer and an output layer; the decoder layer includes an integrated attention mechanism, a feedforward neural network and an encoder-decoder attention mechanism.

6. A retrieval method based on spatiotemporal knowledge graph attention network according to claim 5, characterized in that: The weight calculation formula of the encoder-decoder attention mechanism is: Among them, i and j both represent natural numbers. represents the attention weight of the jth position of the encoder output when the decoder generates the i-th output, represents the similarity score between the i-th output of the decoder and the j-th output of the encoder, k represents the position index in the encoder output sequence, T represents the total length of the encoder output sequence, represents the similarity score between the i-th output of the decoder and the k-th output of the encoder.

7. A retrieval method based on spatiotemporal knowledge graph attention network according to claim 6, characterized in that: The output layer is expressed as: Among them, O represents the original score of the output layer, Softmax represents the conversion of the output into a probability distribution, Y represents the output of the previous layer, W3 represents the weight of the output layer, and b3 represents the bias of the output layer.

8. According to claim 1, a retrieval method based on spatiotemporal knowledge graph attention network is characterized in that: The knowledge graph ontology is constructed through entities and entity relationships, timestamps and spatial coordinates are assigned to the knowledge graph ontology, and the attribute information between entities and entity relationships is combined to construct a spatiotemporal knowledge graph.

9. A retrieval device based on a spatiotemporal knowledge graph attention network, characterized in that: include: The spatiotemporal knowledge graph construction module obtains the search statement, extracts the entities and entity relationships in the search statement, assigns timestamps and spatial coordinates to the entities and entity relationships, and combines the attribute information between the entities and entity relationships to construct the spatiotemporal knowledge graph; The attention network model building module builds an attention network model based on the encoder-decoder framework, where the attention mechanism in the encoder and decoder is replaced by an integrated attention mechanism; The retrieval module inputs the timestamps, spatial coordinates, and attribute information of entities and entity relationships in the spatiotemporal knowledge graph into the encoder to generate a high-dimensional feature vector. The decoder decodes and outputs the high-dimensional feature vector to obtain the retrieval result. The integrated attention mechanism is expressed as: Among them, CombinedAttention represents the integrated attention mechanism, Q represents the query vector, K represents the key vector, V represents the value vector, P represents the position encoding vector, g represents the gating function, Concat represents the connection operation, i, j and k all represent natural numbers, head i represents the output of the i-th head, R represents the learnable parameters, represents the output weight matrix, n represents the total number of key vectors, exp represents the exponential function, sim represents the dot product similarity function, represents the weight matrix of the i-th head used to transform the query vector, represents the j-th key vector, represents the weight matrix used to transform the key vector of the i-th head, represents the position encoding vector of the jth element, represents the sigmoid activation function, represents the kth key vector, represents the positional encoding vector of the kth element, represents the j-th value vector, Represents the weight matrix used to transform the value vector of the i-th head.

Images