Method and device for converting natural language problem into multi-model data query statement

Abstract

The invention provides a method and device for converting a natural language question into a multi-model data query statement, and relates to the technical field of artificial intelligence. The method comprises the steps that the natural language question input by a user is determined; obtaining a corresponding mode item based on the natural language question, and inputting the natural language question and the mode item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is obtained by training a sample natural language problem carrying a first real label and a sample mode item; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; and compiling the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement. A natural language can be efficiently and accurately converted into the multi-model data query statement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a natural language question to multi-model data query statement conversion method and device. BACKGROUND

[0002] With the rapid development of big data technology, the data relied on by modern software applications presents a multi-model heterogeneous characteristic. To cope with this challenge, multi-model databases (MMDB) have emerged, which can uniformly manage multiple data models such as documents, graphs, and key values. As the core interface for users to interact with multi-model databases, the syntax structure of multi-model query language (MMQL) is significantly heterogeneous, which brings new challenges for users to efficiently operate the database.

[0003] Currently, the industry mainly uses supervised deep learning-based methods for natural language to query language conversion tasks, but these technologies are mainly designed for structured query language (SQL). Since MMQL has essential differences in syntax structure, semantic checking mechanism, etc. from SQL, existing methods cannot be directly migrated to natural language to MMQL conversion tasks. In addition, the schema representation of multi-model data is significantly different from traditional structured data, making existing schema encoding techniques not directly applicable to the multi-model data environment. Therefore, how to efficiently and accurately convert natural language to MMQL has become a key technical problem that needs to be solved in the field of multi-model databases. SUMMARY

[0004] To solve the problems in the prior art, the present application provides a natural language question to multi-model data query statement conversion method and device.

[0005] The present application provides a natural language question to multi-model data query statement conversion method, comprising: determining a natural language question input by a user; based on the natural language question, obtaining a corresponding schema item, inputting the natural language question and the schema item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on sample natural language questions carrying first real labels and sample schema items; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; compiling and processing the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement.

[0006] The method for converting a natural language question into a multi-model data query statement provided by the application comprises the following steps: The query statement template is extracted according to the sample multi-model data query statement corresponding to the sample natural language question. The first real label is obtained by forward compiling based on the query statement template and the sample multi-model data query statement; the first real label is the multi-model data query intermediate representation corresponding to the sample multi-model data query statement.

[0007] The method for converting a natural language question into a multi-model data query statement provided by the application comprises the following steps: The multi-model data query intermediate representation is modified and detected based on at least one of the following modification rules: keyword modification, pattern modification, sub-statement rearrangement and symbol matching.

[0008] The method for converting a natural language question into a multi-model data query statement provided by the application comprises the following steps: A structured query language data set is obtained; the structured query language data set comprises structured data and a first statement pair; the first statement pair is a natural language question and a structured query statement pair. The structured data is converted into at least one unstructured data, and multi-model data is obtained based on the structured data and the unstructured data; the first statement pair is rewritten into a natural language question and a target multi-model data query statement pair, and a multi-model query language data set is obtained. The sample natural language question is determined based on the natural language question in the multi-model query language data set, and the sample pattern item is determined based on the target multi-model data query statement pair corresponding to the natural language question.

[0009] The method for converting a natural language question into a multi-model data query statement provided by the application comprises the following steps: An initial embedding representation is determined based on the natural language question and the pattern item of the multi-model data pattern. The initial embedding representation is input into a graph neural network to obtain a candidate pattern item output by the graph neural network and an occurrence probability corresponding to the candidate pattern item; the graph neural network is trained based on a sample initial embedding representation carrying a second real label. determine the mode item corresponding to the natural language question from the candidate mode items based on a set probability range.

[0010] According to the natural language question to multi-model data query statement conversion method provided by the application, the training based on the sample initial embedding representation carrying the second real label comprises: input the sample initial embedding representation into an original graph neural network to obtain candidate items output by the original graph neural network and appearance probabilities corresponding to the candidate mode items; determine a loss function value of the candidate mode items and the second real label, adjust parameters of the original graph neural network according to the loss function value, determine direct connection relationships of each node in the graph structure data model, and adjust the appearance probabilities corresponding to the candidate mode items according to the direct connection relationships; when the original graph neural network converges, obtain the graph neural network.

[0011] The application further provides a natural language question to multi-model data query statement conversion device, comprising: an input information determination module configured to determine a natural language question input by a user; an intermediate representation generation module configured to obtain a corresponding mode item based on the natural language question, input the natural language question and the mode item into a conversion model, and obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on sample natural language questions carrying first real labels and sample mode items; a query statement template determination module configured to determine a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; a query statement generation module configured to compile and process the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement.

[0012] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the natural language question to multi-model data query statement conversion method as described above when executing the computer program.

[0013] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the natural language question to multi-model data query statement conversion method as described above.

[0014] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the natural language question to multi-model data query statement conversion method of any of the above.

[0015] Compared with directly outputting the multi-model data query statement by the model, the natural language question to multi-model data query statement conversion method and device provided by the embodiment of the application can obtain the multi-model data query intermediate representation by inputting the natural language question and the corresponding schema item into the conversion model, and can obtain the multi-model data query statement by compiling and processing the multi-model data query intermediate representation according to the target query statement template, can strip the syntax details of the specific multi-model data query language, focus on the general logical structure of the multi-model data query, and obtain the multi-model data query statement in combination with the corresponding target query statement template, and can efficiently and accurately convert the natural language into the multi-model data query statement. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 is one of the flowcharts of the natural language question to multi-model data query statement conversion method provided by the application.

[0018] Figure 2 is the second flowchart of the natural language question to multi-model data query statement conversion method provided by the application.

[0019] Figure 3 is the structural schematic diagram of the natural language question to multi-model data query statement conversion device provided by the application.

[0020] Figure 4 is the structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present application more clear, the technical scheme of the present application will be described clearly and completely in the following combined with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0022] The following will be described in combination with Figures 1-4The application discloses a conversion method and device of a natural language question into a multi-model data query statement.

[0023] Figure 1 Figure 1 is a flowchart of a conversion method of a natural language question into a multi-model data query statement provided by the application. Figure 1 As shown in the figure, the method comprises the following steps. Step 101: determining a natural language question input by a user.

[0024] The natural language question refers to a statement or question for querying data expressed in a natural language such as Chinese or English and used in daily life. For example, "querying the number of Iranian actors participating in the movies of 'Jim Jarmusch'" is a natural language question.

[0025] The mode item refers to a basic unit for describing a data structure in a multi-model database. For example, the mode item can be a label of a node or an edge in a graph model, or a collection name or attribute name of a document object.

[0026] In the embodiment, the natural language question input by the user can be determined by text input or voice input, and the embodiment does not make further limitation.

[0027] Step 102: obtaining a corresponding mode item based on the natural language question, inputting the natural language question and the mode item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is obtained by training based on a sample natural language question carrying a first real label and a sample mode item.

[0028] The multi-model data query intermediate representation refers to a query operation sequence capable of reflecting the query logic of different multi-model data query languages without depending on a specific multi-model data query language.

[0029] Step 103: determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation.

[0030] The query statement template refers to a template for obtaining a query statement by mapping a query operation to a corresponding syntax structure. The target query statement template is one of the at least two preset query statement templates corresponding to the multi-model data query intermediate representation.

[0031] Step 104: performing a compilation process on the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement.

[0032] The multi-model data query statement refers to a specific query instruction written based on a specific multi-model data query language. The multi-model data query language can be ArangoDB Query Language (AQL) for multi-model database ArangoDB, can be Equivalent Combination of Query Languages (ECQL), or can be SQL++ applied to multi-model database OrientDB, etc.

[0033] Exemplarily, the natural language question "How many movies did Ronald Reagan once participate in?" and the schema item "Movie[G], Movie[R], Movie[R].id, Movie[G].Cast, Actor[G], Actor[R], Actor[R].id, Actor[G].Cast" can be input into the conversion model to obtain the multi-model data query intermediate representation output by the conversion model "query operations TableScan, Filter, Project, NodeScan, Expand, CrossJoin, Aggregate, and Return, and corresponding operands actor, actor.name = 'Ronald Reagan', actor.id, (actor:Actor), (actor)-[r:Cast]->(movie:Movie), actor.id = actor[R].id, COUNT(*) AS cnt, and cnt"; According to the target query statement template, the multi-model data query statement is obtained by compiling processing: MATCH(actor:Actor)-[r:Cast)->(mowie:Movie) WHERE actor.id = { SELECT actor.id FROM actor WHERE actor.name = 'Ronald Reagan' } WITH COUNT(*) AS cnt RETURN cnt Compared with directly outputting a multi-model data query statement by a model, the conversion method from a natural language question to a multi-model data query statement provided in the embodiment of the application can obtain a multi-model data query intermediate representation by inputting a natural language question and a corresponding pattern item into a conversion model, and can obtain a multi-model data query statement by compiling and processing the multi-model data query intermediate representation according to a target query statement template, so that the multi-model data query language syntax details can be stripped, and the general logic structure of the multi-model data query can be focused on, and the multi-model data query statement can be obtained in combination with the corresponding target query statement template, so that the natural language can be efficiently and accurately converted into the multi-model data query statement.

[0034] The query language conversion method based on the large language model relies on the pre-learning of the large language model on the natural query language and the converted target query language, but currently, there are few related public corpora of multi-model data queries, and most of the public data sets of multi-model data queries are missing, and the public data sets of multi-model data queries that can be obtained also contain mixed structured data query languages such as SQL, so that the large language model cannot effectively learn the syntax structure and other features of the corresponding multi-model data query from the data set, and the large language model will misuse the keywords in other query languages when converting the query statement.

[0035] To solve this technical problem, based on the above embodiment, before training based on the sample natural language question carrying the first real label and the sample pattern item, the method further comprises: obtaining a structured query language data set; the structured query language data set comprises structured data and a first statement pair; the first statement pair is a natural language question and a structured query statement pair; converting the structured data into at least one unstructured data, obtaining multi-model data based on the structured data and the unstructured data; rewriting the first statement pair into a natural language question and a target multi-model data query statement pair to obtain a multi-model query language data set; determining the sample natural language question based on the natural language question in the multi-model query language data set, and determining the sample pattern item based on the target multi-model data query statement pair corresponding to the natural language question.

[0036] Specifically, the structured query language data set can be an SQL structured data set, and the unstructured data set can be a graph structured data set and a document structured data set. It can be understood that the specific data structure data set given in the embodiment is for the purpose of facilitating the understanding of the application scheme, and is not intended to limit the scope of the application scheme.

[0037] Exemplarily, when converting the structured data into the graph structure data, the entity table can be converted into the node type data of the graph structure data, and the edge table can be converted into the edge type data of the graph structure data. Specifically, first, a graph node class label same as the entity table can be created, a corresponding graph node can be created based on each tuple in the entity table, the label of the graph node is the graph node class label, the value of the unique identification key corresponds to the primary key attribute value of the tuple, and the attribute values of the remaining columns of the tuple correspond to each other; then a graph edge class label same as the edge table can be created, the corresponding start graph node and end graph node can be found based on the foreign key value of each tuple in the edge table, a new edge can be created, the start point of the created new edge corresponds to the start graph node just found, the end point corresponds to the end graph node just found, and the attribute values of the remaining columns of the edge correspond to each other except the start point and end point foreign key columns in the tuple.

[0038] When converting the structured data into the document structure data, the entity table and the edge table under the many-to-one relationship can be converted into the nested document data. Specifically, a new attribute can be added to each document object in the document collection, the name of the new attribute is equal to the table name of the edge table, and the value of the new attribute is an empty list; for each tuple in the edge table, a plurality of corresponding tuples can be found in the entity table according to the foreign key value of the tuple; for each tuple in the entity table that meets the condition, a document object can be added to the list just created, and the attribute values of the document object correspond to each other.

[0039] In other words, in the embodiment, the multi-model query language data set includes two parts of data and query statement pairs. Specifically, the data is multi-model data including structured data and unstructured data, and the query statement pair is a natural language question and a target multi-model data query statement pair. Exemplarily, after determining the sample natural language question, the sample pattern item can be obtained by parsing the syntax structure of the corresponding multi-model data query statement in the query statement pair.

[0040] The original conversion model is a deep learning model that captures language rules from a large amount of text data through large-scale unsupervised learning. The conversion model can be obtained by pre-training through a large-scale general corpus and fine-tuning through a multi-model query language data set. The pre-training stage does not require manual annotation, and the conversion task of the natural language question to the multi-model data query intermediate representation can be adapted by fine-tuning a small amount of labeled data based on the pre-training.

[0041] Exemplarily, the multi-model query language data set in the embodiment includes 127 training samples, each training sample can include a natural language question and a multi-model data query statement, and the pattern item and the multi-model data query intermediate representation can be obtained through the multi-model data query statement.

[0042] Before training based on the natural language question carrying the first real label and the pattern item, the method further comprises: According to the sample natural language question corresponding to the sample multi-model data query statement, the query statement template is extracted; Based on the query statement template and the sample multi-model data query statement, a first real label is obtained by forward compilation; the first real label is a multi-model data query intermediate representation corresponding to the sample multi-model data query statement.

[0043] Specifically, the query statement template is extracted from the logical query plan generated when the multi-model database parses the sample multi-model data query statement. The logical query plan can be represented as a binary tree, whose nodes are query operations, and each query operation by its operation name , template , number of child nodes . Illustratively, the query statement template can be obtained by a query operation template extractor to extract the sample multi-model data query statement and parse its logical query plan.

[0044] The initial logical query plan can be obtained by calling the query parser of the multi-model database, and then the post-order traversal sequence of the logical query plan is calculated, and the sequence is serialized into text to obtain the corresponding multi-model data query intermediate representation, and each query operation name constitute a sequence, and obtain the query statement skeleton .

[0045] Wherein, the post-order traversal is a depth-first traversal method of binary tree, and the access order is: left subtree, right subtree, root node, which can convert binary tree to sequence.

[0046] Illustratively, according to the target query statement template, the multi-model data query intermediate representation is compiled to obtain a multi-model data query statement, which can include: According to the number of child nodes corresponding to each query operation node in the target query statement template, the multi-model data query intermediate representation is compiled to obtain a multi-model data query statement.

[0047] Specifically, the number of child nodes refers to the number of child nodes connected to each query operation node in the target query statement template.

[0048] In the embodiment, the number of sub-nodes can determine the hierarchical relationship of each query operation node, and the query operation and the corresponding operation number in the intermediate representation are organized according to the hierarchical relationship to compile and process to obtain a multi-model data query statement, so that the lossless of the compilation process at the execution level can be realized, and the accuracy of the obtained multi-model data query statement can be improved.

[0049] To reduce the influence of possible errors in the multi-model data query intermediate representation output by the conversion model on the accuracy of the final multi-model data query statement, based on any of the above embodiments, before the multi-model data query intermediate representation is compiled and processed according to the target query statement template to obtain a multi-model data query statement, the method further comprises: modifying and detecting the multi-model data query intermediate representation based on at least one of the following modification rules: keyword modification, pattern modification, sub-statement rearrangement, and symbol matching.

[0050] Specifically, keyword modification can be replacing a part of the string literal in the string literal with a question keyword whose similarity exceeds a threshold query with the keyword.

[0051] Pattern modification can be as follows: in the path expression of the EXPAND operation (n: SOURCE)-[r: RELATIONSHIP]->(m: TARGET), if the label RELATIONSHIP exists in the global multi-model schema, the direction of the relationship r is modified according to the pattern modification; in the path expression of the EXPAND operation (n: SOURCE)-[r: RELATIONSHIP]->(m: TARGET), if the label RELATIONSHIP does not exist in the global multi-model schema, and there is only one relationship label in the global schema, the label is used to replace RELATIONSHIP; in the path expression of the EXPAND operation (n: SOURCE)-[r: RELATIONSHIP]->(m: TARGET), if there is no any relationship between the SOURCE and TARGET nodes, the sub-statement is removed; in the path expression of the EXPAND operation (n: SOURCE)-[r: RELATIONSHIP]->(m), if there is only one node label in the global multi-model schema, the label is specified for m.

[0052] The sub-statement rearrangement can be removing an empty RETURN sub-statement; removing, in a RETURN sub-statement, an attribute that does not exist in a preceding sub-statement; removing, in a RETURN sub-statement, an attribute that does not conform to the return field syntax rule; moving a RETURN sub-statement to the end of the corresponding sub-query if the RETURN sub-statement does not appear at the end of the sub-query; moving a SORT sub-statement to after a SCAN, EXPAND, or LET sub-statement that introduces a variable of the SORT sub-statement; moving an AGGREGATE sub-statement to after a SCAN, EXPAND, or LET sub-statement that introduces a variable of the AGGREGATE sub-statement; moving a FILTER sub-statement to after a SCAN, EXPAND, or LET sub-statement that introduces a variable of the FILTER sub-statement; adding a RETURN sub-statement at the end of a sub-query if there is no RETURN sub-statement in the sub-query, the RETURN sub-statement returning an attribute that is a final variable appearing in a preceding sub-statement.

[0053] The symbol matching can be correcting a RETURN operation to a FILTER if the RETURN operation contains only one attribute that conforms to the filter condition syntax; removing incomplete filter conditions that cannot be parsed if a FILTER operation contains the incomplete filter conditions; correcting a comparison symbol to EQ if the FILTER operation contains the comparison symbol that cannot be parsed; correcting an invalid sorting rule to DESC if a SORT operation contains the invalid sorting rule that cannot be parsed; and correcting the operation to a RETURN if a starting symbol does not identify any operation and an operand conforms to the RETURN syntax.

[0054] Based on any of the above embodiments, determining the pattern item corresponding to the natural language question comprises: Determining an initial embedding representation based on the natural language question and the pattern items of the multi-model data schema; Inputting the initial embedding representation into a graph neural network to obtain a candidate pattern item output by the graph neural network and an occurrence probability corresponding to the candidate pattern item; the graph neural network is trained based on sample initial embedding representations carrying second true labels; Determining the pattern item corresponding to the natural language question from the candidate pattern items based on a set probability range.

[0055] Specifically, the pattern items of the multi-model data schema can include scope and label information, etc. The scope refers to a specific data model to which a node or an edge belongs, and can describe the semantics and usage range of the node or the edge in the multi-model database.

[0056] It can be understood that the specific value of the set probability range can be set by a person skilled in the art according to actual needs, and the present embodiment does not make further limitation thereon.

[0057] A pattern item sequence can be obtained based on the graph structure data model in the multi-model data model, and a question word sequence can be obtained based on the natural language question. The pattern item sequence and the question word sequence are concatenated to obtain an initial embedding representation. Specifically, the initial embedding representation can be obtained by concatenating the pattern item sequence and the question word sequence using a graph neural network (GNN).

[0058] Graph neural networks are a type of deep learning model specifically designed to process graph-structured data. They can capture the relationships between nodes, the global graph structure, and the feature information of nodes and edges.

[0059] Specifically, the initial embedding representation of the natural language problem can be determined by the following formula : in, represents the embedding function, Representing natural language questions, A schema item representing a multi-model data schema, Indicates the scope of a pattern item, Represents the label name of a pattern item.

[0060] exist In the case of embedding dimension, The shape is .

[0061] in, Representation mode item The number of

[0062] The probability of occurrence of the candidate pattern item output by the graph neural network can be a binary classification probability, and dimension 0 indicates that the pattern item The probability that the corresponding multi-model data query statement does not appear , dimension 1 represents the pattern item in natural language questions The probability of not appearing in the corresponding multi-model data query statement ( ).

[0063] Based on any of the foregoing embodiments, initial embedding representation training based on samples carrying the second true label includes: Inputting the initial embedding representation of the sample into the original graph neural network to obtain the candidate items output by the original graph neural network and the corresponding occurrence probabilities of the candidate pattern items; determine a loss function value of the candidate pattern item and the second real label, adjust parameters of the original graph neural network according to the loss function value; determine a direct connection relationship of each node in the graph structure data model, and adjust the appearance probability corresponding to the candidate pattern item according to the direct connection relationship; determine the graph neural network when the original graph neural network converges.

[0064] Specifically, a sample natural language question and a second real label are determined based on a multi-model query language data set, and a sample initial embedding representation of the sample natural language question is determined based on a graph structure data model in a multi-model data pattern. It can be understood that the second real label refers to a sample pattern item corresponding to the sample natural language question.

[0065] A negative log likelihood (NLL) function can be used as a loss function to calculate the loss function value of the candidate pattern item and the second real label.

[0066] After adjusting the parameters of the graph neural network according to the loss function value each time, the appearance probability corresponding to the candidate pattern item is adjusted when it is determined that the direct connection relationship of each node in the graph structure data model meets one of the following two conditions.

[0067] (1) represents an entity, represents an attribute of the entity; (2) and both represent an entity, and there is another node connected to and .

[0068] wherein, and are directly connected neighbor nodes.

[0069] The appearance probability corresponding to the corrected candidate pattern item can be obtained by the following formula : Figure 2 is the architecture diagram of the natural language question to multi-model data query statement conversion method provided by the present application, as Figure 2 shown, in order to specifically explain the function of the natural language question to multi-model data query statement conversion method provided by the present embodiment, a specific example is provided below.

[0070] In the preprocessing stage, a structured query language dataset is obtained; the structured query language dataset includes structured data and a first sentence pair; the first sentence pair is a natural language question and a structured query sentence pair; the structured data is converted into at least one unstructured data, and multi-model data is obtained based on the structured data and the unstructured data; the first sentence pair is rewritten into a natural language question and a target multi-model data query sentence pair to obtain a multi-model query language dataset; a sample natural language question is determined based on the natural language question in the multi-model query language dataset; A sample multi-model data query sentence corresponding to the sample natural language question is obtained from the multi-model database through a query operation template extractor, and then the query sentence template is extracted; a first true label is obtained by a forward compiler based on the query sentence template provided by the query operation template extractor and the sample multi-model data query sentence obtained from the multi-model query language dataset; the first true label is a sample multi-model data query intermediate representation corresponding to the sample multi-model data query sentence; in the training stage, a conversion model can be trained based on the sample natural language question carrying the first true label and the sample pattern item; The sample pattern item is determined based on the target multi-model data query sentence pair corresponding to the natural language question; it can be understood that the sample pattern item is a second true label, and a sample initial embedding representation of the sample natural language question is determined based on a graph structure data model in a multi-model data pattern; In the training stage, the sample initial embedding representation can be input into an original graph neural network to obtain a candidate item output by the original graph neural network and an occurrence probability corresponding to the candidate pattern item; a loss function value of the candidate pattern item and the second true label is determined, and the parameters of the original graph neural network are adjusted according to the loss function value; the direct connection relationship of each node in the graph structure data model is determined, and the occurrence probability corresponding to the candidate pattern item is adjusted according to the direct connection relationship for path enhancement processing; when the original graph neural network converges, the graph neural network is obtained; In the inference stage, a natural language question input by a user can be determined; a pattern item sequence is determined based on a graph structure data model in a multi-model data pattern, a question token sequence is determined based on a sample natural language question, and the pattern item sequence and the question token sequence are spliced to obtain an initial embedding representation; the initial embedding representation is input into a graph neural network to obtain a candidate pattern item and its occurrence probability output by the graph neural network; the graph neural network is trained based on a sample initial embedding representation carrying a second true label, and a multi-model data pattern item is selected from the candidate pattern item based on a set probability range; inputting the natural language question and the pattern item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the multi-model data query intermediate representation can be modified and detected by a fault-tolerant back compiler based on at least one of the modification rules of keyword modification, pattern modification, sub-sentence rearrangement, and symbol matching; a target query statement template is determined from at least two preset query statement templates based on the multi-model data query intermediate representation; and the multi-model data query intermediate representation is compiled and processed according to the number of sub-nodes corresponding to each query operation node in the target query statement template to obtain a multi-model data query statement, which is returned to the user.

[0071] The multi-model query language data set constructed by the foregoing embodiments is divided into a separate validation set, and verification shows that, in the validation sets of the data sets IMDB-AQL, IMDB-ECQL, and IMDB-SQL++ constructed by the embodiments of the present application, in the case where the base model of the conversion model is T5-base, the logical form (LF) accuracy of the multi-model data query statement obtained by conversion reaches 79.67%, 67.86%, and 69.54% respectively, which is respectively improved by 10.34%, 9.46%, and 46.72% compared with the T5-base model itself, and is respectively improved by 22.82%, 15.84%, and 34.98% compared with the latest PURPLE-GPT4 method based on a large language model in 2024, so it can be seen that the translation accuracy of the natural language question to multi-model data query statement conversion method provided by the embodiments of the present application is obviously improved for various multi-model data query languages.

[0072] In addition, since the syntax structure of the SQL++ query language is quite different from that of the structured query language, and the training corpus related thereto is also less, the translation accuracy of the prior art for the SQL++ query language is significantly lower than that for the other two query languages, while the translation accuracy of the natural language question to multi-model data query statement conversion method provided by the embodiments of the present application for the SQL++ query language remains at the same level as for the other two query languages, so it can be seen that the natural language question to multi-model data query statement conversion method provided by the embodiments of the present application is more robust for different multi-model data query languages.

[0073] The natural language question to multi-model data query statement conversion device provided by the present application is described below, and the natural language question to multi-model data query statement conversion device described below can be correspondingly referred to the natural language question to multi-model data query statement conversion method described above.

[0074] Figure 3 is a structural schematic diagram of the natural language question to multi-model data query statement conversion device provided by the present application, likeFigure 3 The device includes: The input information determination module 310 is configured to determine a natural language question input by a user. The intermediate representation generation module 320 is configured to obtain a corresponding pattern item based on the natural language question, input the natural language question and the pattern item into a conversion model, and obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on a sample natural language question carrying a first real label and a sample pattern item. The query statement template determination module 330 is configured to determine a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation. The query statement generation module 340 is configured to compile and process the multi-model data query intermediate representation based on the target query statement template to obtain a multi-model data query statement.

[0075] Based on any of the above embodiments, the device further includes a preprocessing module configured to extract the query statement template from a sample multi-model data query statement corresponding to the sample natural language question. The first real label is obtained by forward compiling the query statement template and the sample multi-model data query statement; the first real label is a multi-model data query intermediate representation corresponding to the sample multi-model data query statement.

[0076] Based on any of the above embodiments, the device further includes a correction module configured to: The multi-model data query intermediate representation is corrected and detected based on at least one correction rule from among keyword correction, pattern correction, sub-statement rearrangement, and symbol matching.

[0077] Based on any of the above embodiments, the preprocessing module is further configured to: Obtain a structured query language dataset; the structured query language dataset includes structured data and a first statement pair; the first statement pair is a natural language question and a structured query statement pair. Convert the structured data into at least one unstructured data, obtain multi-model data based on the structured data and the unstructured data, rewrite the first statement pair into a natural language question and a target multi-model data query statement pair, and obtain a multi-model query language dataset; Determine the sample natural language question based on a natural language question in the multi-model query language dataset, and determine the sample pattern item based on a target multi-model data query statement pair corresponding to the natural language question.

[0078] Based on any of the above embodiments, the input information determination module 310 is specifically configured to: determining an initial embedding representation based on the natural language question and pattern terms of the multi-modal data pattern; Inputting the initial embedding representation into a graph neural network to obtain candidate pattern items output by the graph neural network and the corresponding occurrence probabilities of the candidate pattern items; the graph neural network is trained based on the initial embedding representation of samples carrying the second true label; The pattern item corresponding to the natural language question is determined from the candidate pattern items based on a set probability range.

[0079] Based on any of the above embodiments, the apparatus further includes a graph neural network training module, configured to: Inputting the initial embedding representation of the sample into the original graph neural network to obtain the candidate items output by the original graph neural network and the corresponding occurrence probabilities of the candidate pattern items; Determine the loss function value of the candidate pattern item and the second true label, and adjust the parameters of the original graph neural network according to the loss function value; determine the direct connection relationship of each node in the graph structure data model, and adjust the occurrence probability corresponding to the candidate pattern item according to the direct connection relationship; When it is determined that the original graph neural network converges, the graph neural network is obtained.

[0080] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communications bus 440. The processor 410 may invoke logic instructions in the memory 430 to execute a method for converting a natural language question into a multi-model data query statement. The method includes: determining a natural language question input by a user and a pattern item corresponding to the natural language question; inputting the natural language question and the pattern item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on sample natural language questions and sample pattern items carrying a first true label; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; and compiling the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement.

[0081] Further, the logic instructions in the memory 430 described above can be implemented in the form of software functional units and sold or used as standalone products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or partially contribute to the prior art, or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0082] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the natural language question to multi-model data query statement conversion method provided by the above-mentioned methods. The method comprises: determining a user input natural language question and a mode item corresponding to the natural language question; inputting the natural language question and the mode item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is obtained based on sample natural language questions carrying first real labels and sample mode items; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; and performing compilation processing on the multi-model data query intermediate representation based on the target query statement template to obtain a multi-model data query statement.

[0083] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the natural language question to multi-model data query statement conversion method provided by the above-mentioned methods. The method comprises: determining a user input natural language question and a mode item corresponding to the natural language question; inputting the natural language question and the mode item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is obtained based on sample natural language questions carrying first real labels and sample mode items; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; and performing compilation processing on the multi-model data query intermediate representation based on the target query statement template to obtain a multi-model data query statement.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0085] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for converting natural language questions into multi-model data query statements, characterized in that: include: Identify natural language questions entered by users; Obtaining corresponding pattern items based on the natural language question, inputting the natural language question and the pattern items into a conversion model, and obtaining a multi-model data query intermediate representation output by the conversion model; The conversion model is trained based on sample natural language questions and sample pattern items carrying a first true label; Determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; The multi-model data query intermediate representation is compiled according to the target query statement template to obtain a multi-model data query statement.

2. The method for converting natural language questions into multi-model data query statements according to claim 1, characterized in that: Before training based on the natural language question and the pattern item carrying the first true label, the method further includes: Extracting the query statement template based on the sample multi-model data query statement corresponding to the sample natural language question; Forward compilation is performed based on the query statement template and the sample multi-model data query statement to obtain a first true label; the first true label is an intermediate representation of the multi-model data query corresponding to the sample multi-model data query statement.

3. The method for converting natural language questions into multi-model data query statements according to claim 1, characterized in that: Before compiling the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement, the method further includes: The multi-model data query intermediate representation is subjected to correction detection based on at least one correction rule selected from keyword correction, pattern correction, sub-statement rearrangement, and symbol matching.

4. The method for converting natural language questions into multi-model data query statements according to claim 1, characterized in that: Before training based on a sample natural language question and a sample pattern item carrying a first true label, the method further includes: Acquire a structured query language dataset; the structured query language dataset includes structured data and a first statement pair, wherein the first statement pair is a natural language question and a structured query statement pair; Converting the structured data into at least one type of unstructured data, and obtaining multi-model data based on the structured data and the unstructured data; rewriting the first statement pair into a natural language question and a target multi-model data query statement pair, to obtain a multi-model query language dataset; The sample natural language question is determined based on the natural language question in the multi-model query language dataset, and the sample pattern item is determined based on the target multi-model data query statement pair corresponding to the natural language question.

5. The method for converting natural language questions into multi-model data query statements according to claim 1, characterized in that: Obtaining corresponding pattern items based on the natural language question includes: determining an initial embedding representation based on the natural language question and pattern terms of the multi-modal data pattern; Inputting the initial embedding representation into a graph neural network to obtain candidate pattern items output by the graph neural network and the corresponding occurrence probabilities of the candidate pattern items; the graph neural network is trained based on the initial embedding representation of samples carrying the second true label; The pattern item corresponding to the natural language question is determined from the candidate pattern items based on a set probability range.

6. The method for converting natural language questions into multi-model data query statements according to claim 5, characterized in that: Training based on sample natural language questions and sample pattern items with first-truth labels, including: Inputting the initial embedding representation of the sample into the original graph neural network to obtain the candidate items output by the original graph neural network and the corresponding occurrence probabilities of the candidate pattern items; Determine the loss function value of the candidate pattern item and the second true label, and adjust the parameters of the original graph neural network according to the loss function value; determine the direct connection relationship of each node in the graph structure data model, and adjust the occurrence probability corresponding to the candidate pattern item according to the direct connection relationship; When it is determined that the original graph neural network converges, the graph neural network is obtained.

7. A device for converting natural language questions into multi-model data query statements, characterized in that: include: An input information determination module, used to determine the natural language question input by the user; an intermediate representation generation module, configured to obtain a corresponding pattern item based on the natural language question, input the natural language question and the pattern item into a conversion model, and obtain an intermediate representation of the multi-model data query output by the conversion model; The conversion model is trained based on sample natural language questions and sample pattern items carrying a first true label; A query statement template determination module, configured to determine a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; The query statement generation module is used to compile the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method for converting a natural language question into a multi-model data query statement according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for converting a natural language question into a multi-model data query statement according to any one of claims 1 to 6 is implemented.

Images