A method, device, equipment and medium for semantic understanding of spatiotemporal questions

Abstract

The present application discloses a method, device, equipment and medium for semantic understanding of spatiotemporal questions, and relates to the field of geographic spatiotemporal query technology. The method comprises: inputting the acquired target natural language spatiotemporal questions into an improved BiLSTM-CRF model to obtain a keyword encoding result, wherein the improved BiLSTM-CRF model adds a multi-head self-attention layer; classifying the target natural language spatiotemporal questions according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal questions; using dependency syntactic analysis to semantically parse the target natural language spatiotemporal questions based on the question type of the target natural language spatiotemporal questions to obtain parsed content; converting the parsed content into an intermediate language; converting the intermediate language into a structured query language, so as to accurately extract the semantic information and intention of the question and realize the conversion from natural language to database query language.

Description

Technical Field

[0001] The present application relates to the field of geographic spatiotemporal query technology, and in particular to a method, device, equipment and medium for semantic understanding of spatiotemporal questions. Background Art

[0002] In today's big data era, more than 18% of massive data is closely related to geographic information, which has led to a significant increase in the demand for geographic spatiotemporal information retrieval. However, most people lack professional knowledge in geographic information retrieval and knowledge bases, and cannot query spatiotemporal related questions in a timely and accurate manner. Current search engines cannot accurately answer questions involving spatiotemporal queries, such as "What are the five restaurants closest to attraction C?", "What subway stations were there in city A in 2010?", "What parks were built in the 19th century within ten kilometers north of attraction B?", etc. In order to solve the above problems and allow more users to experience the convenience of life and work brought by the big data era, it is necessary to convert natural language questions into structured query languages ​​based on semantic understanding and simplify the difficulty of user interaction with data.

[0003] Semantic understanding of natural language questions is a key technology and research hotspot for spatiotemporal query. Current research mainly focuses on semantic parsing of questions and converting natural language into structured query languages ​​that can be processed by computers, such as SQL, SPARQL / GeoSPARQL, Cypher and other formats.

[0004] However, most previous studies have analyzed natural language spatiotemporal questions based on rules, templates, and syntactic patterns. Research on semantic understanding is not sufficient, and the semantic information extraction methods are relatively traditional and cannot accurately identify the intent of the question and valuable information. The resulting semantic retrieval results are of low accuracy. Summary of the invention

[0005] The purpose of this application is to provide a method, device, equipment and medium for semantic understanding of spatiotemporal questions, which can accurately extract the semantic information and intention of questions and realize the conversion from natural language to database query language.

[0006] To achieve the above objectives, this application provides the following solutions.

[0007] In a first aspect, the present application provides a method for semantic understanding of spatiotemporal questions, comprising the following steps.

[0008] Obtain the target natural language spatiotemporal question.

[0009] The target natural language spatiotemporal question is input into a trained deep learning model to obtain a keyword encoding result; the trained deep learning model is an improved BiLSTM-CRF model, and the improved BiLSTM-CRF model adds a multi-head self-attention layer between the BiLSTM layer and the CRF layer in the BiLSTM-CRF model; the multi-head self-attention layer is used to extract the global features of the output results of the BiLSTM layer; the keyword encoding result includes the encoding of several keywords in the target natural language spatiotemporal question.

[0010] The target natural language spatiotemporal question is classified according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal question.

[0011] Dependency syntactic analysis is adopted to perform semantic analysis on the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain analysis content; the analysis content includes the dependency syntactic relationship between geographic entity codes, the query conditions and query targets of the target natural language spatiotemporal question; the geographic entity codes exist in the keyword encoding results.

[0012] The parsed content is converted into an intermediate language.

[0013] The intermediate language is converted into structured query language; structured query language is used to access the database.

[0014] In a second aspect, the present application provides a device for semantic understanding of spatiotemporal questions, comprising the following modules.

[0015] The sentence acquisition module is used to obtain the target natural language spatiotemporal question.

[0016] The keyword extraction module is used to: input the target natural language spatiotemporal question into a trained deep learning model to obtain a keyword encoding result; the trained deep learning model is an improved BiLSTM-CRF model, and the improved BiLSTM-CRF model adds a multi-head self-attention layer between the BiLSTM layer and the CRF layer in the BiLSTM-CRF model; the multi-head self-attention layer is used to extract the global features of the output result of the BiLSTM layer; the keyword encoding result includes the encoding of several keywords in the target natural language spatiotemporal question.

[0017] The classification module is used to classify the target natural language spatiotemporal question according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal question.

[0018] The semantic parsing module is used to: use dependency syntactic analysis to semantically parse the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain parsed content; the parsed content includes the dependency syntactic relationship between geographic entity codes, the query conditions and query targets of the target natural language spatiotemporal question; the geographic entity code exists in the keyword encoding result.

[0019] The intermediate language conversion module is used to convert the parsed content into an intermediate language.

[0020] The structured conversion module is used to convert the intermediate language into a structured query language; the structured query language is used to access the database.

[0021] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for semantic understanding of spatiotemporal questions.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for semantic understanding of spatiotemporal questions.

[0023] According to the specific embodiments provided by the present application, the present application discloses the following technical effects: the present application provides a method, device, equipment and medium for semantic understanding of spatiotemporal questions. On the premise of fully considering the semantic information of spatiotemporal questions, natural language spatiotemporal questions are firstly semantically encoded, and the improved BiLSTM-CRF model is used to realize automatic semantic encoding of questions to extract semantic information in questions, and the spatiotemporal questions are classified based on the keyword encoding results to realize question intention recognition, and dependency syntactic analysis is used to semantically parse the encoding elements of different types of questions to fully understand their intentions, convert the parsed content into an intermediate language, construct an intermediate language structure and convert it into a structured query language, which can accurately extract the semantic information and intention of the question and realize the conversion from natural language to database query language, solving the problem that the existing methods are not sufficiently studied in semantic understanding, the semantic information extraction method is relatively traditional, and the question intention and valuable information cannot be accurately identified, and the semantic retrieval results obtained have low accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 This is a diagram of the application environment of a method for semantic understanding of spatiotemporal questions in one embodiment of the present application.

[0026] Figure 2 A flowchart of a method for semantic understanding of spatiotemporal questions provided in one embodiment of the present application.

[0027] Figure 3 A flowchart of the research idea for semantic understanding of natural language spatiotemporal questions provided in one embodiment of the present application.

[0028] Figure 4 A schematic diagram of the improved BiLSTM-CRF model structure provided in one embodiment of the present application.

[0029] Figure 5 A schematic diagram of the basic framework of a multi-head self-attention mechanism provided in one embodiment of the present application.

[0030] Figure 6 A schematic diagram of a structured query generation process provided in an embodiment of the present application.

[0031] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0034] The method for understanding the semantics of spatiotemporal questions provided in the embodiments of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the target natural language spatiotemporal question to the server 104. After the server 104 receives the target natural language spatiotemporal question, for the target natural language spatiotemporal question, the server 104 inputs the acquired target natural language spatiotemporal question into the improved BiLSTM-CRF model to obtain the keyword encoding result. The improved BiLSTM-CRF model adds a multi-head self-attention layer; classifies the target natural language spatiotemporal question according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal question; uses dependency syntactic analysis to semantically parse the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain the parsed content; converts the parsed content into an intermediate language; converts the intermediate language into a structured query language. The server 104 can feed back the obtained structured query language for the target natural language spatiotemporal question to the terminal 102. In addition, in some embodiments, the spatiotemporal question semantic understanding method can also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 can directly perform spatiotemporal question semantic understanding processing on the target natural language spatiotemporal question, or the server 104 can obtain the target natural language spatiotemporal question from the data storage system and perform spatiotemporal question semantic understanding processing on the target natural language spatiotemporal question.

[0035] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, and IoT devices, and the IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0036] In an exemplary embodiment, Figure 2 As shown, a method for understanding the semantics of spatiotemporal questions is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, and the steps include the following steps 201 to 206.

[0037] Step 201: Obtain a target natural language spatiotemporal question.

[0038] Step 202: Input the target natural language spatiotemporal question into a trained deep learning model to obtain a keyword encoding result; the trained deep learning model is an improved BiLSTM-CRF model, and the improved BiLSTM-CRF model adds a multi-head self-attention layer between the BiLSTM layer and the CRF layer in the BiLSTM-CRF model; the multi-head self-attention layer is used to extract global features of the output result of the BiLSTM layer; the keyword encoding result includes the encoding of several keywords in the target natural language spatiotemporal question.

[0039] Step 203: classify the target natural language spatiotemporal question according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal question.

[0040] Step 204: Dependency syntactic analysis is used to semantically parse the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain parsed content; the parsed content includes the dependency syntactic relationship between geographic entity codes, the query conditions and query targets of the target natural language spatiotemporal question; the geographic entity codes exist in the keyword encoding results.

[0041] Step 205: Convert the parsed content into an intermediate language.

[0042] Step 206: Convert the intermediate language into structured query language; the structured query language is used to access the database.

[0043] Implement the above steps 201 to 206. Under the premise of fully considering the semantic information of spatiotemporal questions, firstly, semantically encode the natural language spatiotemporal questions, and use the improved BiLSTM-CRF model to realize automatic semantic encoding of questions to extract semantic information in questions, classify spatiotemporal questions based on keyword encoding results, realize question intention recognition, use dependency syntactic analysis to semantically parse the encoding elements of different types of questions, fully understand their intentions, convert the parsed content into an intermediate language, construct an intermediate language structure and convert it into a structured query language, which can accurately extract the semantic information and intention of the question, realize the conversion from natural language to database query language, and solve the problem that the existing methods are not sufficient in terms of semantic understanding, the semantic information extraction method is relatively traditional, and the question intention and valuable information cannot be accurately identified, and the semantic retrieval results obtained are of low accuracy. The present application can also fully understand the semantics of questions, realize the mapping of natural language questions to database query questions, and cover basic queries and compound queries related to time and space.

[0044] The research process of semantic understanding of natural language spatiotemporal questions is as follows: Figure 3 shown.

[0045] The target natural language spatiotemporal question in this application is a Chinese spatiotemporal question. Extracting useful semantic information is the key to converting unstructured questions into structured query statements. In order to facilitate computer understanding and processing of natural language spatiotemporal questions, this application represents the semantic information in the target natural language spatiotemporal question with encoding results, and according to the characteristics of Chinese spatiotemporal queries, the existing location-related semantic encoding mode is expanded to accurately represent the semantic information in the target natural language spatiotemporal question.

[0046] The basic semantic coding elements of Chinese spatiotemporal questions are shown in Table 1.

[0047] (1) Question words: These include place questions (where, which, etc.), specific questions (what, etc.), selection questions (which one, which ones, which one, which one, etc.), quantity questions (how many, how many), measurement questions (how big, how long, how far, etc.), and judgment questions (is it, whether, etc.).

[0048] (2) Place name: the name of a geographical entity, such as Shanghai and scenic spot B.

[0049] (3) Place type: classification of geographical entities, such as hospitals and parks.

[0050] (4) Attributes: Attributes of a place, such as its population, area, and time.

[0051] (5) Quality: the quality of the location and its attributes, and the type of location, such as nearest and highest.

[0052] (6) Activity: indicates action, such as through, flow through and build.

[0053] (7) Spatiotemporal relations: Describes spatiotemporal relations, including direction, topology, and time relations, such as northeast, nearby, around, before, and after.

[0054] (8) Numbers and units: including time, distance, and quantity, such as 2021, five kilometers, and three.

[0055] Table 1 Basic semantic coding elements of Chinese spatiotemporal questions

[0056]

[0057] The improved BiLSTM-CRF model in step 202 includes an embedding layer, a BiLSTM layer, a multi-head self-attention layer and a CRF (Conditional Random Fields) layer. The structure of the improved BiLSTM-CRF model is as follows: Figure 4 shown.

[0058] Among them, the embedding layer is used to perform vector conversion on the target natural language spatiotemporal question to obtain an embedding vector set; the embedding vector set includes the embedding vector corresponding to each character in the target natural language spatiotemporal question.

[0059] The BiLSTM layer is used to extract features from the embedded vector set to obtain text semantic information features.

[0060] The multi-head self-attention layer is used to extract global features of text semantic information features to obtain global features.

[0061] The CRF layer is used to determine the keyword encoding results based on global features.

[0062] The semantic encoding of Chinese spatiotemporal questions can be regarded as a sequence labeling process. Training on the labeled data can achieve automatic encoding of question semantics. Chinese spatiotemporal questions contain rich semantic information, the sentence structure has long-distance dependencies, and the semantic encoding elements are diverse. There is a phenomenon that the same word is encoded as different types of information in different contexts.

[0063] Therefore, this application integrates the multi-head attention mechanism based on the BiLSTM-CRF model to obtain an improved BiLSTM-CRF model (BiLSTM-MUSA-CRF), which uses multiple attention heads (attention head 1, attention head 2, ... attention head n) to calculate the weights of different positions respectively, so as to better understand the semantic relationship between different words in a sentence, capture long-distance dependencies and resolve ambiguity.

[0064] The embedding layer converts the original input text sequence (target natural language spatiotemporal question) into a digital vector and inputs it into the BiLSTM layer and the multi-head self-attention layer in sequence. The BiLSTM layer is used to learn the long-term dependencies and contextual information of words in the target natural language spatiotemporal question, and the multi-head self-attention layer is used to obtain the global feature representation of the input text sequence and the correlation between each word; finally, the CRF layer is used to obtain the label prediction result of the sequence, that is, the keyword encoding result.

[0065] 1) The embedding layer converts discrete text data (all character texts of the target natural language spatiotemporal question) into a continuous, low-dimensional vector representation. Input text , the embedding layer will generate Learn a corresponding embedding vector. Output vector ,in Is the corresponding character text These embedding vectors are used as input to the BiLSTM layer.

[0066] 2) The BiLSTM layer receives the output vector of the embedding layer and processes the input sequence bidirectionally, taking into account the contextual information of the current moment and before and after it, thereby effectively capturing the semantic features in the sequence. The hidden state of BiLSTM encodes rich sequence information, including word meaning, grammatical structure, and context, providing important feature representation for subsequent label prediction. Through the forget gate and input gate of the gating mechanism, BiLSTM can handle long-distance dependencies and alleviate the gradient vanishing or exploding problem faced by traditional RNN.

[0067] 3) Multi-head self-attention layer: In sequence labeling tasks, the traditional BiLSTM-CRF model needs to gradually establish dependencies and cannot fully reflect the global information of sentences and word characteristics. During the training process, the BiLSTM-CRF model may lose some features and information. The multi-head attention mechanism has better global dependency modeling, feature extraction and long-range dependency modeling capabilities. At the same time, by calculating the attention weights of multiple heads in parallel, it can speed up the calculation process and is more efficient than the standard attention mechanism.

[0068] like Figure 5 As shown in the figure, each feature vector in the input sequence will undergo three linear transformations to obtain the query vector, key vector, and value vector. For each attention head, the query vector and the key vector are calculated by dot product to obtain the original attention score; to prevent the gradient from exploding or disappearing, the obtained score is normalized and softmax is applied to convert it into the corresponding probability distribution. , as shown in formula (1).

[0069] (1).

[0070] in, Represent the query vector, key vector, and value vector respectively. Indicates the dimension of the key vector.

[0071] Each value vector is weighted and summed according to the corresponding attention weight to obtain the attention representation of each head; the attention representation concatenated by the concat layer passes through the linear transformation layer to obtain the multi-head attention representation Yt; finally, after an additional linear transformation layer and activation function layer, the calculation result of the multi-head attention mechanism is output, that is, the global feature is obtained.

[0072] 4) The role of the CRF layer in the improved BiLSTM-CRF model is to effectively model label dependencies and ensure that the generated label sequence is coherent and reasonable. The CRF layer globally normalizes the label sequence output by the multi-head attention layer and jointly trains and adjusts the model prediction results, taking into account the dependencies between labels and improving model performance. By maximizing the log-likelihood function learning parameters, the CRF layer enables the improved BiLSTM-CRF model to more accurately capture label relationships and generate reasonable and coherent label sequences, thereby enhancing the accuracy and coherence of sequence labeling tasks.

[0073] According to the above description, taking "Which parks within ten kilometers north of scenic spot B were built in the 19th century?" as an example, after being input into the improved BiLSTM-CRF model, it can be encoded as "pprd3tds". Among them, pp represents 'scenic spot B', r represents 'north', d represents 'ten kilometers', 3 represents a question sentence, t represents 'park', d represents 'nineteenth century', and s represents 'built'.

[0074] Step 203 specifically includes: inputting the keywords corresponding to the keyword encoding result and the target natural language spatiotemporal question into the trained classification model to obtain the question type of the target natural language spatiotemporal question. The trained classification model is a model trained with the sample natural language spatiotemporal sentence and the keywords corresponding to the keyword encoding result in the sample natural language spatiotemporal sentence as input, and the sample question type of the sample natural language spatiotemporal sentence as a label.

[0075] The trained classification model is a model obtained by training a linear multi-classification support vector machine.

[0076] Question classification helps to better understand the user's intentions and needs, and then provide customized query results based on different categories of questions. This application uses GeoQuestions201, the gold standard dataset for geospatial question answering, as a benchmark, and expands question types such as direction, distance, and time, including basic queries, compound queries, and fuzzy queries, and reclassifies questions. In the context of geographic space and time, the answers expected to be returned by spatiotemporal question queries are related to factors such as location, distance, direction, quantity, and maximum value. Therefore, questions are classified according to the expected answer type, and the question types include location queries, distance queries, direction queries, buffer queries, count queries, maximum value queries, list queries, and judgment queries. Question types and their examples are shown in Table 2.

[0077] (1) Location query: directly inquire about the specific location of a place, or the location of a place that has a geographic spatial relationship with a certain place.

[0078] (2) Distance query: Query the distance between two locations.

[0079] (3) Direction query: asking about the direction of a certain location relative to another location.

[0080] (4) Buffer zone query: inquires about the location type within a certain range of a certain location (at a certain time), including fuzzy queries such as "around", "near", and "next to" the location.

[0081] (5) Count query: ask how many places of a specific type there are in a certain place (at a certain time).

[0082] (6) Maximum value query: Query the maximum or minimum value of a feature at a certain location (at a certain time), or return the location with the maximum value feature.

[0083] (7) Judgment query: asking whether a certain location (at a certain time) meets certain conditions or attributes, or asking whether there is a geographical and temporal relationship between two locations, etc.

[0084] These classification methods provide a structured processing framework for spatiotemporal questions, making it easier to accurately understand user needs and provide corresponding query results.

[0085] Table 2 Question types and examples

[0086]

[0087] Scenic spots B and C are famous scenic spots.

[0088] After semantic encoding, the extracted keywords can be used as important features to distinguish question categories. Therefore, in the question classification stage, a linear multi-classification support vector machine (LinearSVC) is used as a classifier to distinguish some compound queries with similar text descriptions, such as "Where is the nearest hotel to attraction C" and "Which hotel is the nearest to attraction C?" The target natural language spatiotemporal question and the keywords such as the interrogative words extracted by encoding are used as input, and the final classification result is obtained after being processed by the trained classification model, that is, the question type of the target natural language spatiotemporal question.

[0089] Input the keywords and target natural language space-time questions corresponding to the keyword encoding results into the trained classification model, specifically including: numerically representing the keywords and target natural language space-time questions corresponding to the keyword encoding results as vectors to obtain numerical vector data; inputting the numerical vector data into the trained classification model. Among them, TF-IDF can be used to numerically represent the keywords and target natural language space-time questions corresponding to the keyword encoding results as vectors. The numerical vector corresponding to the target input text obtained by TF-IDF is used as the input of the trained classification model, and the target input text is the keywords and target natural language space-time questions corresponding to the keyword encoding results.

[0090] Step 204 specifically includes: using the LTP tool and dependency syntax analysis to semantically parse the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain parsed content.

[0091] On the basis of question classification, semantic analysis of questions within the category is continued. The main purpose is to identify the relationship between the extracted geographic entity codes, obtain the query conditions for each type of question, and locate the specific query target, providing a basis for the next step of query conversion. In the questions in the field of geographic spatiotemporal query, the main focus is on the dependency syntactic relationship between geographic entities. Dependency syntactic relationships include spatial relationships, temporal relationships, and syntactic dependency relationships. LTP tools are used to perform dependency syntactic analysis on questions. According to the needs and characteristics of different questions, the subject-predicate relationship, verb-object relationship, and attributive-predicate relationship in the questions are identified, and the dependency syntactic relationship between geographic entity codes and the query target of the questions are extracted.

[0092] 1) Spatial relationship: The three elements of spatial description are location object (P), reference object (C) and spatial relationship (r). Location object is the core of spatial description, describing the object being located or described. Reference object is the benchmark or reference point used to locate location object in spatial description, which provides a reference framework for location object. Spatial relationship is the spatial connection between location object and reference object, including direction relationship (such as east, west, south, north, northeast, northwest, southeast, southwest), distance relationship (such as surrounding, nearby, etc.) and topological relationship (such as bordering, containing, adjacent, etc.). The spatial relationship triple in the question is expressed as follows (2).

[0093] (2).

[0094] In the formula, Indicates the target natural language spatiotemporal question All objects that have a relationship, Represents all spatial relations contained in the target natural language spatiotemporal question.

[0095] For example, the sentence "Parks and banks to the north near the museum" contains , , , .

[0096] According to the semantic encoding in the target natural language spatiotemporal question, if there is a spatial relationship Encoding, based on dependency syntax and rules, identifies the location object and reference object of the spatial relationship and determines the spatial relationship triple. The spatial relationship in the spatial relationship triple determines the query method and conditions of the target natural language spatiotemporal question. Fuzzy queries such as distance relationships, such as surrounding and nearby, require the definition of a buffer query range.

[0097] 2) Temporal relations: Considering the temporal factors when describing geographic entities, such as the historical evolution, change trends, and status of geographic entities in a specific period, this method can more comprehensively understand and describe geographic phenomena and help analyze the changes in geographic data in the temporal dimension. The temporal relations in the target natural language spatiotemporal question mainly focus on the temporal attributes of geographic entities or the temporal relations between geographic entities. If the time code is recognized in the target natural language spatiotemporal question, the time is assigned as an attribute or relationship to the geographic entity code in the target natural language spatiotemporal question, and subsequent queries are performed based on this time condition.

[0098] 3) Syntactic relations: The syntactic relations of geographical entities in the target natural language spatiotemporal questions include subject-predicate, verb-object, parallel, attributive-predicate, preposition-object and other dependency relations. For example, if there are geographical entities in parallel relations in the target natural language spatiotemporal questions, and the first geographical entity is associated with a certain relation (such as spatial relation), then the geographical entities in parallel with it are usually also associated with the same relation; if the dependency relation between geographical entities is attributive-predicate relation, it indicates that there is a modification relation between geographical entities.

[0099] Determine the query target of natural language spatiotemporal questions based on syntactic relations: location queries usually take place p or place type t as the subject; distance queries take place p as the subject; direction queries need to determine the reference object and query object based on the subject-predicate relationship, verb-object relationship, etc. in the dependency relationship; the query target of buffer, count, maximum value, and list queries is generally place type t; the query target of judgment queries is the IF function.

[0100] In another exemplary embodiment of the present application, in order to resolve the significant differences between natural language and database query language so that the GIS system can more efficiently process and execute user query requests, the present application introduces an intermediate language (Intermediate Representation, IR) as a bridge, constructs the intermediate language based on the semantic understanding of the question, and then converts the intermediate language into the structured query language required by the database.

[0101] IR is the basis for generating structured query statements in databases. Its design should accurately reflect the semantics of natural language expressions and be easily converted into structured query statements. The IR structure of spatiotemporal query consists of functions, relations, logical words, etc., including modules such as question categories, geographic entities and relations, query conditions, and return values. IR lists the extracted geographic entity coding classes and defines the relationship between geographic entities as a function; it uses the query target of the question as a variable. , the rest of the information is used as constants; query conditions such as buffer, direction, distance, sorting, fuzzy query, topology query, etc. are defined as functions, such as Buffer (p, number, t), Direction ( ,East, )、 , Order (o, number), etc.

[0102] Taking "Which parks within ten kilometers north of scenic spot B were built in the 19th century?" as an example, the target natural language spatiotemporal question is converted into an intermediate language representation according to the following steps.

[0103] (1) Extracting geographic entities and relationships: Extracting based on the identified coding classes, such as places (p) and their relationships, place types (t), attributes (o), etc.

[0104] (2) Declare terms: declare based on the coding class. For example, declare p('B attraction') means declare B attraction as a place (p); declare t(hotel) means declare hotel as a place type (t).

[0105] (3) Define the query target: Use the unknown query target as a variable: Assign the query target variable a value , i.e., location type .

[0106] (4) Define functions: Define and reference functions based on the query conditions of the sentence. For example, Buffer (B Scenic Spot, 3000m, Hotel): defines the buffer spatial relationship, indicating the "hotel" within the buffer range of "B Scenic Spot"; Direction (B Scenic Spot, North, Hotel): defines the spatial direction relationship, indicating the relationship between the location type and the location's north direction.

[0107] (5) Constructing logical representation: First, use the question category as the beginning to define the overall framework; declare the extracted geographic entities and express the relationships with functions; define the query target and assign variables; define the function and fill in the parameters; and finally return the variable value.

[0108] The middle language of the example sentence "Which parks within ten kilometers north of attraction B were built in the 19th century?" is as follows.

[0109] LIST(x):p('B Scenic Spot') AND p('A City') AND InCity('B Scenic Spot', 'A City') AND t(Hotel) AND x=t AND Direction('B Scenic Spot''North', x) AND Buffer('B Scenic Spot',3000m, x) ANDReturn(x).

[0110] LIST( ): p('B attractions') AND p('A city') AND InCity('B attractions', 'A city') AND t(hotel) AND =t AND Direction('B Scenic Spot', 'North', ) AND Buffer ('B attraction', 3000m, ) AND o(time,<,2000) AND Return( ).

[0111] Structured query language is used to access databases, and Cypher query language is a declarative query language widely used in graph database interaction. Therefore, this application takes Cypher language as an example to introduce the algorithm process of generating structured query language from intermediate language.

[0112] The basic structure of Cypher query statements includes MATCH, WHERE, RETURN, ORDER BY, WITH, etc. Among them, MATCH matches the information such as nodes and relationships in the knowledge base, that is, the geographic entities and relationships of the intermediate language; the WHERE statement is generated by the function module (query condition) of the intermediate language, and the function module is written by the cypher statement. For example, the direction function in the query condition is calculated by the inverse tangent function to obtain the direction and degree results such as east, west, south, north, northeast, southeast, northwest, southwest, southwest by south, and southwest by west; RETURN means returning the answer to the question, which is determined by the question type and the query target of the intermediate language.

[0113] like Figure 6 As shown, the main algorithm for generating a Cypher statement includes the following steps 301 to 303.

[0114] Step 301: The MATCH statement matches the geographic entity portion of the IR and assigns a unique variable to each extracted geographic concept, such as the matched place node is defined as n and the matched label (place type) is t.

[0115] Step 3011: When there is a relationship between two locations or between a location and a type, a corresponding relationship model and direction are established.

[0116] Step 3012: When there are multiple geographic entities in the IR (such as place name n and place name type t), use the MATCH statement to match them in sequence and connect them with commas.

[0117] Step 302: If there is a function in the IR, add a WHERE clause to express the query condition, and map the query condition of the IR to the corresponding function (distance, direction function) and parameters.

[0118] Step 3021: When there are multiple query conditions, they are connected with AND to indicate that the query conditions are met at the same time.

[0119] Step 3022: When the query condition is sorting, a WITH clause needs to be added before the sorting function to directly pass the matching nodes to subsequent operations; or a RETURN clause needs to be added before the sorting function.

[0120] If the function does not exist in the IR, proceed to step 301.

[0121] Step 303: The RETURN statement returns the variable of the query target according to the question type and the query target in the IR. When returning a judgment question as an answer, you can use the function "COUNT( )>0" or CASE statements, etc. return a Boolean value.

[0122] The effect of the spatiotemporal question semantic understanding method proposed in this application is verified below.

[0123] Due to the lack of Chinese spatiotemporal question datasets, this application mainly uses the gold standard geospatial dataset GeoQuestions201 and geographic questions crawled from the network platform, and performs geographic entity replacement, sentence reconstruction and synonym replacement on these data to establish and expand the Chinese dataset. The specific situation of the dataset is as follows.

[0124] (1) GeoQuestions201: As the gold standard dataset for geospatial queries, it contains 201 English geospatial questions, including location, spatial relationships of geographic entities, spatial relationships of geographic features, quantity, and aggregation questions.

[0125] (2) Spatiotemporal semantic coding dataset: Based on the gold standard dataset of geospatial queries, the query sentences are expanded to include 3486 questions, 13 coding types, and a total of 51688 characters. The number of semantic codes of each type is shown in Table 3.

[0126] (3) Classification dataset: There are 9414 geospatial questions in the dataset, and the question categories are manually annotated.

[0127] Table 3 Statistics of spatiotemporal semantic coding dataset

[0128]

[0129] The parameters of the improved BiLSTM-CRF model were adjusted, the number of self-attention heads in the multi-head attention layer was set to 12, and the Adam optimization algorithm was used with a learning rate of 0.001. In order to avoid overfitting, early-stop and dropout mechanisms were applied. The batch parameter of the model was set to 64, and the number of epochs was 60. The spatiotemporal semantic coding dataset was divided into training set, validation set, and test set according to the ratio of 8:1:1. The specific corpus statistics are shown in Table 4. The dataset is annotated in BIOES format (B indicates the beginning of the annotation sequence, I indicates the middle, E indicates the end, S indicates a single word, and O indicates irrelevant information).

[0130] Table 4 Statistics of training corpus

[0131]

[0132] In the comparative experiment, the BiLSTM model and the BiLSTM-CR model F are selected to verify the performance of the improved BiLSTM-CRF model proposed in this application in the semantic encoding task. The model performance is evaluated using indicators commonly used in sequence labeling tasks, including precision, recall and F1 value. The specific results are shown in Table 5.

[0133] Table 5 Comparison results of model experiments

[0134]

[0135] It can be concluded from Table 5 that the improved BiLSTM-CRF model has improved accuracy by 6.32% and 2.71% respectively, recall by 6.52% and 2.63%, and F1 value by 6.18% and 2.53% compared with the baseline models BiLSTM and BiLSTM-CRF. According to the experimental results of the improved model, it can be concluded that the encoding accuracy of question words is high, and the accuracy of location and spatiotemporal relationship extraction is slightly lower than that of other encodings.

[0136] The natural language questions are converted into Cypher language. Using the existing method, 160 questions are randomly selected by category in the data set, and each question type contains 20 questions. The corresponding Cypher queries are manually written for these questions. The manually generated Cypher queries are compared with the Cypher queries generated by the method of this application, using both automatic evaluation and manual evaluation methods.

[0137] (1) Automatic evaluation. If the Cypher query generated by the present application is exactly the same as the Cypher query string manually generated by the existing method, the generated SAPRQL query is considered to be correct.

[0138] (2) Manual evaluation: Manually determine whether the generated Cypher query is semantically identical to the manually generated Cypher query. If they are identical, the generated query is considered correct.

[0139] According to statistics, the accuracy of automatic evaluation is 56%, and the accuracy of manual evaluation is 71%. Some experimental results are shown in Tables 6, 7 and 8 below.

[0140] Table 6 Experimental results example

[0141]

[0142] Table 7 Experimental results example

[0143]

[0144] Table 8 Experimental results example

[0145]

[0146] In response to the problems of insufficient semantic understanding of questions, incomplete extraction of semantic information, inaccurate intent recognition, etc. in current natural language spatiotemporal queries, this application proposes a natural language-oriented spatiotemporal question semantic understanding method. Experimental results show that the natural language spatiotemporal semantic understanding and conversion method proposed in this application can accurately extract the semantic information and intent of questions and realize the conversion from natural language to database query language. This application helps to improve the user experience of geographic information systems and promote intelligent services of geographic spatial information. Semantic understanding oriented to natural language enables computers to better understand user intent and provides corresponding interfaces for users to query, thereby improving the user-friendliness, query expression capabilities and human-computer interaction level of the database system.

[0147] The present application also provides an application scenario, which applies the above-mentioned spatiotemporal question semantic understanding method. Specifically: The spatiotemporal question semantic understanding method provided in this embodiment can be applied in geographic information query scenarios. The geographic information query scenario includes a geographic information input link and a spatiotemporal question semantic understanding link; the target natural language spatiotemporal question enters the spatiotemporal question semantic understanding link from the geographic information input link, and obtains the corresponding structured query language. The spatiotemporal question semantic understanding method provided in this embodiment belongs to the spatiotemporal question semantic understanding link. Specifically, in the process of semantic understanding link of spatiotemporal questions in videos, the acquired target natural language spatiotemporal questions can be input into the improved BiLSTM-CRF model to obtain keyword encoding results. The improved BiLSTM-CRF model adds a multi-head self-attention layer; the target natural language spatiotemporal questions are classified according to the keyword encoding results to obtain the question types of the target natural language spatiotemporal questions; dependency syntactic analysis is used to semantically parse the target natural language spatiotemporal questions based on the question types of the target natural language spatiotemporal questions to obtain parsed content; the parsed content is converted into an intermediate language; and the intermediate language is converted into a structured query language.

[0148] Based on the same inventive concept, the embodiment of the present application also provides a spatiotemporal question semantic understanding device for implementing the spatiotemporal question semantic understanding method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more spatiotemporal question semantic understanding device embodiments provided below can refer to the limitations of the spatiotemporal question semantic understanding method above, and will not be repeated here.

[0149] In an exemplary embodiment, a device for understanding spatiotemporal question semantics is provided, including the following modules.

[0150] The sentence acquisition module T1 is used to: acquire the target natural language spatiotemporal question.

[0151] The keyword extraction module T2 is used to: input the target natural language spatiotemporal question into a trained deep learning model to obtain a keyword encoding result; the trained deep learning model is an improved BiLSTM-CRF model, and the improved BiLSTM-CRF model adds a multi-head self-attention layer between the BiLSTM layer and the CRF layer in the BiLSTM-CRF model; the multi-head self-attention layer is used to extract the global features of the output results of the BiLSTM layer; the keyword encoding result includes the encoding of several keywords in the target natural language spatiotemporal question.

[0152] The classification module T3 is used to classify the target natural language spatiotemporal question according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal question.

[0153] The semantic parsing module T4 is used to: use dependency syntactic analysis to semantically parse the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain parsed content; the parsed content includes the dependency syntactic relationship between geographic entity codes, the query conditions and query targets of the target natural language spatiotemporal question; the geographic entity code exists in the keyword encoding result.

[0154] The intermediate language conversion module T5 is used to convert the parsed content into an intermediate language.

[0155] The structured conversion module T6 is used to convert the intermediate language into a structured query language; the structured query language is used to access the database.

[0156] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store semantic processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for semantic understanding of spatiotemporal questions is implemented.

[0157] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0159] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0162] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0163] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0164] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for semantic understanding of spatiotemporal questions, characterized in that: The method for semantic understanding of spatiotemporal questions comprises: Obtaining target natural language spatiotemporal question sentences; The target natural language spatiotemporal question is input into a trained deep learning model to obtain a keyword encoding result; the trained deep learning model is an improved BiLSTM-CRF model, and the improved BiLSTM-CRF model adds a multi-head self-attention layer between the BiLSTM layer and the CRF layer in the BiLSTM-CRF model; the multi-head self-attention layer is used to extract the global features of the output result of the BiLSTM layer; the keyword encoding result includes the encoding of several keywords in the target natural language spatiotemporal question; Classifying the target natural language spatiotemporal question according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal question; Using dependency syntactic analysis, semantically parsing the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain parsed content; the parsed content includes dependency syntactic relations between geographic entity codes, query conditions and query targets of the target natural language spatiotemporal question; the geographic entity code exists in the keyword encoding result; Converting the parsed content into an intermediate language specifically includes: Extract geographic entities and relationships: Extract according to the identified coding class; Declare terms: declare according to the coding class; Define the query target: use the unknown query target as a variable; Define functions: define and reference functions based on the query conditions of the sentence: define buffer space relationships and define spatial direction relationships; Constructing logical representation: Start with the question category and define the overall framework; declare the extracted geographic entities and express the relationships with functions; define the query target and assign variables; define the function and fill in the parameters; and finally return the variable value. The intermediate language is converted into structured query language; structured query language is used to access the database.

2. The method for semantic understanding of spatiotemporal questions according to claim 1, characterized in that: The improved BiLSTM-CRF model includes an embedding layer, a BiLSTM layer, a multi-head self-attention layer and a CRF layer; The embedding layer is used to perform vector conversion on the target natural language spatiotemporal question to obtain an embedding vector set; the embedding vector set includes the embedding vector corresponding to each character in the target natural language spatiotemporal question; The BiLSTM layer is used to extract features from the embedded vector set to obtain text semantic information features; The multi-head self-attention layer is used to extract global features from text semantic information features to obtain global features; The CRF layer is used to determine the keyword encoding results based on global features.

3. The method for semantic understanding of spatiotemporal questions according to claim 1, characterized in that: The target natural language spatiotemporal question is classified according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal question, which specifically includes: The keywords corresponding to the keyword encoding results and the target natural language spatiotemporal question are input into the trained classification model to obtain the question type of the target natural language spatiotemporal question.

4. The method for semantic understanding of spatiotemporal questions according to claim 3, characterized in that: The trained classification model is a model obtained by training a linear multi-classification support vector machine.

5. The method for semantic understanding of spatiotemporal questions according to claim 1, characterized in that: The question types include location query, distance query, direction query, buffer query, count query, maximum value query, list query and judgment query.

6. The method for semantic understanding of spatiotemporal questions according to claim 1, characterized in that: Dependency syntactic analysis is used to semantically parse the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain parsed content, which specifically includes: The target natural language spatiotemporal question is semantically parsed based on the question type of the target natural language spatiotemporal question by using the LTP tool and dependency syntactic analysis to obtain parsed content.

7. The method for semantic understanding of spatiotemporal questions according to claim 3, characterized in that: Input the keywords corresponding to the keyword encoding results and the target natural language spatiotemporal question into the trained classification model, including: Numerical vector representation is performed on the keywords and target natural language spatiotemporal question sentences corresponding to the keyword encoding results to obtain numerical vector data; Input the digitized vector data into the trained classification model.

8. A device for understanding the semantics of spatiotemporal questions, characterized in that: The spatiotemporal question semantic understanding device comprises: A sentence acquisition module is used to: acquire target natural language spatiotemporal question sentences; A keyword extraction module is used to: input the target natural language spatiotemporal question into a trained deep learning model to obtain a keyword encoding result; the trained deep learning model is an improved BiLSTM-CRF model, and the improved BiLSTM-CRF model adds a multi-head self-attention layer between the BiLSTM layer and the CRF layer in the BiLSTM-CRF model; the multi-head self-attention layer is used to extract the global features of the output result of the BiLSTM layer; the keyword encoding result includes the encoding of several keywords in the target natural language spatiotemporal question; A classification module, used to: classify the target natural language spatiotemporal question according to the keyword encoding result to obtain the question type of the target natural language spatiotemporal question; A semantic parsing module, for: using dependency syntactic analysis, semantically parsing the target natural language spatiotemporal question based on the question type of the target natural language spatiotemporal question to obtain parsed content; the parsed content includes the dependency syntactic relationship between geographic entity codes, the query condition and the query target of the target natural language spatiotemporal question; the geographic entity code exists in the keyword encoding result; The intermediate language conversion module is used to convert the parsed content into an intermediate language, specifically including: Extract geographic entities and relationships: Extract according to the identified coding class; Declare terms: declare according to the coding class; Define the query target: use the unknown query target as a variable; Define functions: define and reference functions based on the query conditions of the sentence: define buffer space relationships and define spatial direction relationships; Constructing logical representation: Start with the question category and define the overall framework; declare the extracted geographic entities and express the relationships with functions; define the query target and assign variables; define the function and fill in the parameters; and finally return the variable value. The structured conversion module is used to convert the intermediate language into a structured query language; the structured query language is used to access the database.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the spatiotemporal question semantic understanding method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for semantic understanding of spatiotemporal questions described in any one of claims 1 to 7 is implemented.

Images