Method and apparatus for converting natural language question into multi-model data query statement

Abstract

The application provides a natural language question to multi-model data query statement conversion method and device, and relates to the technical field of artificial intelligence, the method comprises the following steps: determining a natural language question input by a user; obtaining a corresponding mode item based on the natural language question; inputting the natural language question and the mode item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is obtained based on sample natural language questions carrying first real labels and sample mode items; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; and performing compilation processing on the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement, which can efficiently and accurately convert natural language into a multi-model data query statement.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for converting natural language questions into multi-model data query statements. Background Technology

[0002] With the rapid development of big data technology, the data upon which modern software applications rely exhibits a multi-model heterogeneous characteristic. To address this challenge, Multi-Model Databases (MMDBs) have emerged, enabling unified management of various data models such as documents, graphs, and key-value pairs. As the core interface for user interaction with MMDBs, the Multi-Model Query Language (MMQL) exhibits significant heterogeneity in its syntax, presenting new challenges for users to efficiently operate the database.

[0003] Currently, the industry primarily employs supervised deep learning-based methods for converting natural language to query language (MMQ), but these techniques are mainly designed for Structured Query Language (SQL). Due to fundamental differences between MMQ and SQL in terms of syntax, semantic checking mechanisms, existing methods are difficult to directly transfer to MMQ conversion tasks. Furthermore, the schema representation of multi-model data differs significantly from traditional structured data, rendering existing schema encoding techniques unsuitable for multi-model data environments. Therefore, achieving efficient and accurate conversion of natural language to MMQ has become a critical technical challenge that urgently needs to be addressed in the field of multi-model databases. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention provides a method and apparatus for converting natural language questions into multi-model data query statements.

[0005] This invention provides a method for converting natural language questions into multi-model data query statements, including: Determine the natural language problem in the user input; Based on the natural language question, the corresponding pattern term is obtained. The natural language question and the pattern term are input into the conversion model to obtain the multi-model data query intermediate representation output by the conversion model. The conversion model is trained based on sample natural language questions and sample pattern terms carrying the first real label. Based on the multi-model data query intermediate representation, the target query statement template is determined from at least two preset query statement templates; The multi-model data query statement is obtained by compiling the intermediate representation of the multi-model data query based on the target query statement template.

[0006] According to the present invention, a method for converting a natural language question into a multi-model data query statement, prior to training based on the natural language question carrying a first real label and the pattern term, the method further includes: The query statement template is extracted from the multi-model data query statement corresponding to the sample natural language question. The first real label is obtained by forward compilation based on the query statement template and the sample multi-model data query statement; the first real label is the intermediate representation of the multi-model data query corresponding to the sample multi-model data query statement.

[0007] According to the method for converting a natural language question into a multi-model data query statement provided by the present invention, before compiling the intermediate representation of the multi-model data query according to the target query statement template to obtain the multi-model data query statement, the method further includes: The intermediate representation of the multi-model data query is corrected and detected based on at least one of the correction rules: keyword correction, pattern correction, sub-statement rearrangement, and symbol matching.

[0008] According to the method for converting natural language questions into multi-model data query statements provided by the present invention, before training based on sample natural language questions and sample pattern terms carrying first true labels, the method further includes: Obtain a structured query language dataset; the structured query language dataset includes structured data and a first statement pair; the first statement pair is a natural language question and a structured query statement pair; The structured data is converted into at least one type of unstructured data, and multi-model data is obtained based on the structured data and the unstructured data; the first statement pair is rewritten into a natural language question and target multi-model data query statement pair to obtain a multi-model query language dataset. The sample natural language question is determined based on the natural language question in the multi-model query language dataset, and the sample pattern item is determined based on the target multi-model data query statement pair corresponding to the natural language question.

[0009] According to the present invention, a method for converting a natural language question into a multi-model data query statement includes determining the pattern item corresponding to the natural language question, comprising: The initial embedding representation is determined based on the pattern terms of the natural language problem and the multi-model data pattern. The initial embedding representation is input into a graph neural network to obtain candidate pattern items and their corresponding occurrence probabilities. The graph neural network is trained based on the initial embedding representation of the sample carrying a second real label. The pattern item corresponding to the natural language problem is determined from the candidate pattern items based on a set probability range.

[0010] According to the present invention, a method for converting natural language questions into multi-model data query statements, based on initial embedding representation training of samples carrying second true labels, includes: The initial embedding representation of the sample is input into the original graph neural network to obtain the candidate items output by the original graph neural network and the occurrence probability corresponding to the candidate pattern items; Determine the loss function values ​​of the candidate pattern item and the second true label, and adjust the parameters of the original graph neural network according to the loss function values; determine the direct connection relationship of each node in the graph structure data model, and adjust the occurrence probability corresponding to the candidate pattern item according to the direct connection relationship; The graph neural network is obtained when the original graph neural network is determined to be converged.

[0011] The present invention also provides a device for converting natural language questions into multi-model data query statements, comprising: The input information determination module is used to determine the natural language questions input by the user; An intermediate representation generation module is used to obtain the corresponding pattern item based on the natural language question, input the natural language question and the pattern item into the conversion model, and obtain the multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on sample natural language questions and sample pattern items carrying a first real label; The query statement template determination module is used to determine the target query statement template from at least two preset query statement templates based on the intermediate representation of the multi-model data query. The query statement generation module is used to compile and process the intermediate representation of the multi-model data query according to the target query statement template to obtain the multi-model data query statement.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for converting natural language problems into multi-model data query statements as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for converting natural language problems into multi-model data query statements as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a method for converting natural language problems into multi-model data query statements as described above.

[0015] Compared to directly outputting multi-model data query statements through a model, the method and apparatus for converting natural language questions into multi-model data query statements provided in this embodiment of the invention obtains an intermediate representation of multi-model data queries by inputting the natural language question and the corresponding pattern item into a conversion model. The intermediate representation of multi-model data queries is then compiled according to the target query statement template to obtain the multi-model data query statement. This method can strip away the syntactic details of specific multi-model data query languages, focus on the general logical structure of multi-model data queries, and combine the corresponding target query statement template to obtain the multi-model data query statement. This method can efficiently and accurately convert natural language into multi-model data query statements. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is one of the flowcharts illustrating the method for converting natural language questions into multi-model data query statements provided by this invention.

[0018] Figure 2 This is the second flowchart illustrating the method for converting natural language questions into multi-model data query statements provided by this invention.

[0019] Figure 3 This is a schematic diagram of the structure of the device for converting natural language questions into multi-model data query statements provided by the present invention.

[0020] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] The following is combined Figures 1-4The present invention describes a method and apparatus for converting natural language questions into multi-model data query statements.

[0023] Figure 1 This is one of the flowcharts illustrating the method for converting natural language questions into multi-model data query statements provided by this invention, such as... Figure 1 As shown, the method includes the following: Step 101: Determine the natural language problem input by the user.

[0024] Natural language problems refer to statements or questions expressed in everyday natural languages ​​such as Chinese and English, used for querying data. For example, "query the number of Iranian actors in 'Jim Jarmusch' movies" is a natural language problem.

[0025] A schema item refers to the basic unit used to describe the data structure in a multi-model database. For example, a schema item can be the label of a node or edge in a graph model, or the collection name or attribute name of a document object.

[0026] In this embodiment, the problem may be determined by the user through text input or by the user through voice input, etc. This embodiment does not further limit the problem in this way.

[0027] Step 102: Obtain the corresponding pattern item based on the natural language question, input the natural language question and the pattern item into the conversion model, and obtain the multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on the sample natural language question and sample pattern item carrying the first real label.

[0028] Multi-model data query intermediate representation refers to a sequence of query operations that does not depend on a specific multi-model data query language and can reflect the query logic of different multi-model data query languages.

[0029] Step 103: Based on the multi-model data query intermediate representation, determine the target query statement template from at least two preset query statement templates.

[0030] A query statement template is a template that maps query operations to corresponding syntax structures to obtain a query statement. The target query statement template is one of at least two pre-defined query statement templates that corresponds to the intermediate representation of the multi-model data query.

[0031] Step 104: Compile the intermediate representation of the multi-model data query according to the target query statement template to obtain the multi-model data query statement.

[0032] Multi-model data query statements refer to specific query instructions written using a particular multi-model data query language. These multi-model data query languages ​​can be ArangoDB Query Language (AQL) for the multi-model database ArangoDB, Equivalent Combination of Query Languages ​​(ECQL), or SQL++ for the multi-model database OrientDB, etc.

[0033] For example, the natural language question "How many movies has Ronald Reagan starred in?" and the pattern items "Movie[G], Movie[R], Movie[R].id, Movie[G].Cast, Actor[G], Actor[R], Actor[R].id, Actor[G].Cast" can be input into the transformation model to obtain the multi-model data query intermediate representation output by the transformation model, which includes "query operations TableScan, Filter, Project, NodeScan, Expand, CrossJoin, Aggregate, and Return, as well as the corresponding operands actor, actor.name = 'Ronald Reagan', actor.id, (actor:Actor), (actor)-[r:Cast]->(movie:Movie), actor.id = actor[R].id, COUNT(*) AS cnt and cnt". The multi-model data query statement is obtained by compiling and processing the target query statement template: MATCH(actor:Actor)-[r:Cast)->(mowie:Movie) WHERE actor.id = { SELECT actor.id FROM actor WHERE actor.name = 'Ronald Reagan' } WITH COUNT(*) AS cnt RETURN cnt Compared to directly outputting multi-model data query statements through a model, the natural language question to multi-model data query statement conversion method provided in this embodiment of the invention obtains a multi-model data query intermediate representation by inputting the natural language question and the corresponding pattern item into the conversion model, and compiles the multi-model data query intermediate representation according to the target query statement template to obtain the multi-model data query statement. This method can strip away the syntactic details of the specific multi-model data query language, focus on the general logical structure of multi-model data query, and obtain the multi-model data query statement by combining it with the corresponding target query statement template. This method can efficiently and accurately convert natural language into multi-model data query statements.

[0034] Query language transformation methods based on large language models rely on the large language model's pre-learning of the natural query language and the target query language. However, currently, there are very few publicly available corpora related to multi-model data queries, most of the public datasets for multi-model data queries are missing, and the available public datasets for multi-model data queries also contain data mixed with structured data query languages ​​such as SQL. This makes it impossible for the large language model to effectively learn the syntactic structure and other features of the corresponding multi-model data queries from the dataset, resulting in the large language model misusing keywords from other query languages ​​when transforming query statements.

[0035] To address this technical problem, based on the above embodiments, before training on the sample natural language question and sample pattern terms carrying the first real label, the method further includes: Obtain a structured query language dataset; the structured query language dataset includes structured data and a first statement pair; the first statement pair is a natural language question and a structured query statement pair; The structured data is converted into at least one type of unstructured data, and multi-model data is obtained based on the structured data and the unstructured data; the first statement pair is rewritten into a natural language question and target multi-model data query statement pair to obtain a multi-model query language dataset. The sample natural language question is determined based on the natural language question in the multi-model query language dataset, and the sample pattern item is determined based on the target multi-model data query statement pair corresponding to the natural language question.

[0036] Specifically, the structured query language dataset can be an SQL structured dataset, and the unstructured dataset can be a graph structured dataset or a document structured dataset, etc. It is understood that the specific data structures of the datasets given in this embodiment are for the purpose of facilitating understanding of the present invention, and not for limiting the scope of the present invention.

[0037] For example, when converting structured data into graph structured data, entity tables can be converted into node-type data of graph structured data, and edge tables can be converted into edge-type data of graph structured data. Specifically, firstly, a graph node class label with the same name as the entity table can be created. Based on each tuple in the entity table, a corresponding graph node is created, and its label is the graph node class label. The value of its unique identification key corresponds to the primary key attribute value of the tuple, and the other attribute values ​​correspond one-to-one with the attribute values ​​of each remaining column of the tuple. Then, a graph edge class label with the same name as the edge table can be created. Based on the foreign key value of each tuple in the edge table, the corresponding start graph node and end graph node are found to create a new edge. The start point of the newly created edge corresponds to the start graph node that was just found, and the end point corresponds to the end graph node that was just found. The attribute values ​​of the edge correspond one-to-one with the attribute values ​​of the remaining columns in the tuple except for the foreign key columns of the start and end points.

[0038] When converting structured data into document structure data, entity tables and edge tables under many-to-one relationships can be transformed into nested document data. Specifically, a new attribute is added to each document object in the document collection, with its name equal to the table name of the edge table and its value an empty list; for each tuple in the edge table, several corresponding tuples are found in the entity table based on its foreign key value; for each tuple in the entity table that meets the conditions, a new document object is added to the list just created, with its attribute values ​​corresponding one-to-one with the attribute values ​​of each column in that tuple.

[0039] In other words, in this embodiment, the multi-model query language dataset includes two parts: data and query statement pairs. Specifically, the data is multi-model data, including structured and unstructured data, and the query statement pairs are natural language questions and target multi-model data query statement pairs. For example, after determining the sample natural language question, the sample pattern item can be obtained by parsing the syntactic structure of the corresponding multi-model data query statement in the query statement pair.

[0040] The original conversion model is a deep learning model that captures language patterns from a large amount of text data through large-scale unsupervised learning. It is first pre-trained on a large-scale general corpus, and then fine-tuned using a multi-model query language dataset to obtain the conversion model. No manual annotation is required in the pre-training stage. Based on the pre-training, a small amount of labeled data can be fine-tuned to adapt to the conversion task of natural language problems to intermediate representations of multi-model data queries in this embodiment.

[0041] For example, in this embodiment, the multi-model query language dataset includes 127 training samples. Each training sample may include natural language questions and multi-model data query statements. Pattern items and intermediate representations of multi-model data queries can be obtained through the statements in the multi-model data query.

[0042] Prior to training based on the natural language question carrying the first real label and the pattern term, the method further includes: The query statement template is extracted from the multi-model data query statement corresponding to the sample natural language question. The first real label is obtained by forward compilation based on the query statement template and the sample multi-model data query statement; the first real label is the intermediate representation of the multi-model data query corresponding to the sample multi-model data query statement.

[0043] Specifically, the query statement template is extracted from the logical query plan generated when parsing sample multi-model data query statements in a multi-model database. The logical query plan can be represented as a binary tree, where each node represents a query operation. By its operation name ,template Number of child nodes Composition. For example, the query statement template can be obtained by acquiring sample multi-model data query statements through a query operation template extractor and parsing their logical query plans.

[0044] The system can invoke the multi-model database's built-in query parser to obtain an initial logical query plan, then calculate the post-traversal sequence of this logical query plan, and serialize it into text to obtain the corresponding intermediate representation of the multi-model data query. Simultaneously, it can extract the name of each query operation. Construct a sequence to obtain the query statement skeleton. .

[0045] Postorder traversal is a depth-first traversal method for binary trees, and the access order is: left subtree, right subtree, root node, which can convert a binary tree into a sequence.

[0046] For example, compiling the intermediate representation of the multi-model data query based on the target query statement template to obtain the multi-model data query statement may include: Based on the number of child nodes corresponding to each query operation node in the target query statement template, the intermediate representation of the multi-model data query is compiled to obtain the multi-model data query statement.

[0047] Specifically, the number of child nodes refers to the number of child nodes connected to each query operation node in the target query statement template.

[0048] In this embodiment, obtaining the number of child nodes can determine the hierarchical relationship of each query operation node. Based on the hierarchical relationship, the query operations and corresponding operands in the intermediate representation are organized and compiled to obtain a multi-model data query statement. This can achieve lossless compilation at the execution level and improve the accuracy of the obtained multi-model data query statement.

[0049] To reduce the impact of potential errors in the intermediate representation of the multi-model data query output by the conversion model on the accuracy of the final multi-model data query statement, based on any of the above embodiments, before compiling the intermediate representation of the multi-model data query according to the target query statement template to obtain the multi-model data query statement, the method further includes: The intermediate representation of the multi-model data query is corrected and detected based on at least one of the correction rules: keyword correction, pattern correction, sub-statement rearrangement, and symbol matching.

[0050] Specifically, keyword correction can involve modifying string literals that contain keywords related to the problem keywords. The similarity exceeds the threshold query Replace the part with this keyword.

[0051] Schema modification can be performed as follows: In the path expression (n:SOURCE)-[r:RELATIONSHIP]->(m:TARGET) of the EXPAND operation, if the label RELATIONSHIP exists in the global multi-model schema, the direction of relation r is modified according to the schema; In the path expression (n:SOURCE)-[r:RELATIONSHIP]->(m:TARGET) of the EXPAND operation, if the label RELATIONSHIP does not exist in the global multi-model schema, and only one relation label exists in the global schema, then RELATIONSHIP is replaced with that label; In the path expression (n:SOURCE)-[r:RELATIONSHIP]->(m:TARGET) of the EXPAND operation, if there is no relation between the SOURCE and TARGET nodes, then the sub-statement is removed; In the path expression (n:SOURCE)-[r:RELATIONSHIP]->(m) of the EXPAND operation, if only one node label exists in the global multi-model schema, then that label is specified for m.

[0052] Subquery rearrangement can include: removing empty RETURN subqueries; removing attributes that do not exist in preceding subqueries within a RETURN subquery; removing attributes that do not conform to the syntax rules of the returned field within a RETURN subquery; moving a RETURN subquery to the end of its corresponding subquery if it does not appear at the end; moving a SORT subquery after a SCAN, EXPAND, or LET subquery that introduces its variable; moving an AGGREGATE subquery after a SCAN, EXPAND, or LET subquery that introduces its variable; moving a FILTER subquery after a SCAN, EXPAND, or LET subquery that introduces its variable; and adding a RETURN subquery at the end of a subquery if it does not contain any RETURN subqueries, returning the final variable that appears in the preceding subqueries.

[0053] Symbol matching can be as follows: if a RETURN operation contains only one property that matches the filter syntax, then the operation is corrected to FILTER; if a FILTER operation contains incomplete filter conditions that cannot be resolved, then these symbols are removed; if a FILTER operation contains an unresolved comparison symbol, then the symbol is corrected to EQ; if a SORT operation contains an invalid collation that cannot be resolved, then the collation is corrected to DESC; if the start symbol does not identify any operation and the operands match the RETURN syntax, then the operation is corrected to RETURN.

[0054] Based on any of the above embodiments, determining the pattern item corresponding to the natural language question includes: The initial embedding representation is determined based on the pattern terms of the natural language problem and the multi-model data pattern. The initial embedding representation is input into a graph neural network to obtain candidate pattern items and their corresponding occurrence probabilities. The graph neural network is trained based on the initial embedding representation of the sample carrying a second real label. The pattern item corresponding to the natural language problem is determined from the candidate pattern items based on a set probability range.

[0055] Specifically, the schema items of a multi-model data schema may include scope and label information. Scope refers to the specific data model to which a node or edge belongs, describing the semantics and usage range of the node or edge in the multi-model database.

[0056] It is understood that those skilled in the art can set the specific value of the probability range according to actual needs, and this embodiment does not impose further limitations on this.

[0057] The pattern item sequence can be obtained based on the graph structure data model in the multi-model data pattern, and the question word sequence can be obtained based on the natural language question. The initial embedding representation is obtained by concatenating the pattern item sequence and the question word sequence. In particular, the initial embedding representation can be obtained by concatenating the pattern item sequence and the question word sequence through a graph neural network (GNN).

[0058] Graph neural networks are a class of deep learning models specifically designed to process graph-structured data. They can capture the relationships between nodes, the global graph structure, and the feature information of nodes and edges.

[0059] Specifically, the initial embedding representation of a natural language problem can be determined using the following formula. : in, Indicates an embedded function. This relates to natural language issues. A schema item representing a multi-model data schema. Indicates the scope of the pattern item. Indicates the label name of the pattern item.

[0060] exist In the case of representing the embedding dimension, The shape is .

[0061] in, Indicates pattern item The quantity.

[0062] The probability of occurrence of a candidate pattern term output by a graph neural network can be a binary classification probability, where dimension 0 represents the probability of occurrence of the pattern term in a natural language problem. The probability of not appearing in the corresponding multi-model data query statement Dimension 1 represents the pattern term in natural language problems. The probability of not appearing in the corresponding multi-model data query statement ( ).

[0063] Based on any of the above embodiments, training based on the initial embedding representation of the sample carrying the second real label includes: The initial embedding representation of the sample is input into the original graph neural network to obtain the candidate items output by the original graph neural network and the occurrence probability corresponding to the candidate pattern items; Determine the loss function values ​​of the candidate pattern item and the second true label, and adjust the parameters of the original graph neural network according to the loss function values; determine the direct connection relationship of each node in the graph structure data model, and adjust the occurrence probability corresponding to the candidate pattern item according to the direct connection relationship; The graph neural network is obtained when the original graph neural network is determined to be converged.

[0064] Specifically, the natural language question and second true label of the sample are determined based on the multi-model query language dataset, and the initial embedding representation of the natural language question of the sample is determined based on the graph structure data model in the multi-model data pattern. It can be understood that the second true label refers to the sample pattern item corresponding to the natural language question of the sample.

[0065] The negative log similarity (NLL) function can be used as the loss function to calculate the loss function value of the candidate pattern item and the second true label.

[0066] Each time the parameters of the graph neural network are adjusted based on the loss function value, the probability of occurrence of the candidate pattern item is adjusted if the direct connection relationship of each node in the graph structure data model meets one of the following two conditions.

[0067] (1) Represents an entity, This represents an attribute of the entity; (2) and Both represent entities, and there exists another node. ,and and All are connected by edges.

[0068] in, and These are directly connected neighbor nodes.

[0069] The probability of occurrence of the corrected candidate pattern item can be obtained using the following formula. : Figure 2 This is a schematic diagram of the architecture of the method for converting natural language questions into multi-model data query statements provided by the present invention, as shown below. Figure 2 As shown, in order to illustrate the functionality of the natural language question to multi-model data query statement conversion method provided in this implementation, a specific example is provided below.

[0070] In the preprocessing stage, a structured query language dataset is obtained; the structured query language dataset includes structured data and a first statement pair; the first statement pair is a natural language question and a structured query statement pair; the structured data is converted into at least one type of unstructured data, and multi-model data is obtained based on the structured data and the unstructured data; the first statement pair is rewritten into a natural language question and a target multi-model data query statement pair to obtain a multi-model query language dataset; the sample natural language question is determined based on the natural language questions in the multi-model query language dataset; The query operation template extractor obtains the sample multi-model data query statements corresponding to the sample natural language questions from the multi-model database, and then extracts the query statement template. The forward compiler performs forward compilation based on the query statement template provided by the query operation template extractor and the sample multi-model data query statements obtained from the multi-model query language dataset to obtain the first true label. The first true label is the intermediate representation of the sample multi-model data query corresponding to the sample multi-model data query statement. During the training phase, the transformation model can be trained based on the sample natural language questions and sample pattern items carrying the first true label. The sample pattern item is determined based on the target multi-model data query statement corresponding to the natural language problem; it can be understood that the sample pattern item is the second true label, and the initial embedding representation of the sample natural language problem is determined based on the graph structure data model in the multi-model data pattern. During the training phase, the samples can be initially embedded into the representation input of the original graph neural network to obtain the candidate items output by the original graph neural network and the occurrence probability corresponding to the candidate pattern items; the loss function value of the candidate pattern items and the second true label is determined, and the parameters of the original graph neural network are adjusted according to the loss function value; the direct connection relationship of each node in the graph structure data model is determined, and the occurrence probability corresponding to the candidate pattern items is adjusted according to the direct connection relationship for path enhancement processing; if the original graph neural network converges, the graph neural network is obtained. During the inference phase, the natural language question input by the user can be determined; a sequence of pattern terms is determined based on the graph structure data model in the multi-model data pattern; a sequence of question terms is determined based on the sample natural language question; the sequence of pattern terms and the sequence of question terms are concatenated to obtain an initial embedding representation; the initial embedding representation is input into a graph neural network to obtain candidate pattern terms and their occurrence probabilities output by the graph neural network; the graph neural network is trained based on the initial embedding representation of the sample carrying a second real label, and multi-model data pattern terms are selected from the candidate pattern terms based on a set probability range; The natural language problem and the pattern term are input into a transformation model to obtain a multi-model data query intermediate representation output by the transformation model. The multi-model data query intermediate representation can be corrected and detected using a fault-tolerant backward compiler based on at least one correction rule among keyword correction, pattern correction, substatement rearrangement, and symbol matching. A target query statement template is determined from at least two preset query statement templates based on the multi-model data query intermediate representation. The multi-model data query intermediate representation is compiled according to the number of child nodes corresponding to each query operation node in the target query statement template to obtain a multi-model data query statement, which is then returned to the user.

[0071] By dividing the multi-model query language dataset constructed in the aforementioned embodiments into a separate validation set, it was verified that in the validation sets of IMDB-AQL, IMDB-ECQL, and IMDB-SQL++ constructed in the embodiments of the present invention, when the base model of the conversion model is T5-base, the logical form (LF) accuracy of the converted multi-model data query statements reached 79.67%, 67.86%, and 69.54%, respectively. Compared with the T5-base model itself, these figures represent improvements of 10.34%, 9.46%, and 46.72%, respectively. Compared with the latest PURPLE-GPT4 method based on a large language model in 2024, these figures represent improvements of 22.82%, 15.84%, and 34.98%, respectively. This demonstrates that the translation accuracy of the natural language problem to multi-model data query statement conversion method provided in the embodiments of the present invention significantly improves the translation accuracy for various multi-model data query languages.

[0072] Furthermore, due to the significant differences in syntax between SQL++ query language and Structured Query Language (SCL), and the limited training corpus associated with it, existing technologies achieve significantly lower translation accuracy for SQL++ query language compared to the other two query languages. In contrast, the natural language problem-to-multi-model data query statement conversion method provided in this invention achieves the same translation accuracy for SQL++ query language as the other two query languages. This demonstrates that the natural language problem-to-multi-model data query statement conversion method provided in this invention is more robust to different multi-model data query languages.

[0073] The following describes the apparatus for converting natural language questions into multi-model data query statements provided by the present invention. The apparatus for converting natural language questions into multi-model data query statements described below can be referred to in correspondence with the method for converting natural language questions into multi-model data query statements described above.

[0074] Figure 3 This is a schematic diagram of the structure of the natural language question to multi-model data query statement conversion device provided by the present invention, as shown below. Figure 3 As shown, the device includes: The input information determination module 310 is used to determine the natural language question input by the user; The intermediate representation generation module 320 is used to obtain the corresponding pattern item based on the natural language question, input the natural language question and the pattern item into the conversion model, and obtain the multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on the sample natural language question and sample pattern item carrying the first real label; The query statement template determination module 330 is used to determine the target query statement template from at least two preset query statement templates based on the intermediate representation of the multi-model data query. The query statement generation module 340 is used to compile and process the intermediate representation of the multi-model data query according to the target query statement template to obtain the multi-model data query statement.

[0075] Based on any of the above embodiments, the device further includes a preprocessing module, used to extract the query statement template based on the sample multi-model data query statement corresponding to the sample natural language problem; The first real label is obtained by forward compilation based on the query statement template and the sample multi-model data query statement; the first real label is the intermediate representation of the multi-model data query corresponding to the sample multi-model data query statement.

[0076] Based on any of the above embodiments, the device further includes a correction module, configured to: The intermediate representation of the multi-model data query is corrected and detected based on at least one of the correction rules: keyword correction, pattern correction, sub-statement rearrangement, and symbol matching.

[0077] Based on any of the above embodiments, the preprocessing module is further configured to: Obtain a structured query language dataset; the structured query language dataset includes structured data and a first statement pair; the first statement pair is a natural language question and a structured query statement pair; The structured data is converted into at least one type of unstructured data, and multi-model data is obtained based on the structured data and the unstructured data; the first statement pair is rewritten into a natural language question and target multi-model data query statement pair to obtain a multi-model query language dataset. The sample natural language question is determined based on the natural language question in the multi-model query language dataset, and the sample pattern item is determined based on the target multi-model data query statement pair corresponding to the natural language question.

[0078] Based on any of the above embodiments, the input information determination module 310 is specifically used for: The initial embedding representation is determined based on the pattern terms of the natural language problem and the multi-model data pattern. The initial embedding representation is input into a graph neural network to obtain candidate pattern items and their corresponding occurrence probabilities. The graph neural network is trained based on the initial embedding representation of the sample carrying a second real label. The pattern item corresponding to the natural language problem is determined from the candidate pattern items based on a set probability range.

[0079] Based on any of the above embodiments, the device further includes a graph neural network training module, used for: The initial embedding representation of the sample is input into the original graph neural network to obtain the candidate items output by the original graph neural network and the occurrence probability corresponding to the candidate pattern items; Determine the loss function values ​​of the candidate pattern item and the second true label, and adjust the parameters of the original graph neural network according to the loss function values; determine the direct connection relationship of each node in the graph structure data model, and adjust the occurrence probability corresponding to the candidate pattern item according to the direct connection relationship; The graph neural network is obtained when the original graph neural network is determined to be converged.

[0080] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for converting a natural language question into a multi-model data query statement. This method includes: determining a natural language question input by a user and a corresponding pattern item; inputting the natural language question and the pattern item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on sample natural language questions and sample pattern items carrying a first real label; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; and compiling the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement.

[0081] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the natural language question to multi-model data query statement conversion method provided by the above methods. The method includes: determining a natural language question input by a user and a pattern item corresponding to the natural language question; inputting the natural language question and the pattern item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model is trained based on sample natural language questions and sample pattern items carrying a first real label; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; and compiling the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement.

[0083] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for converting a natural language question into a multi-model data query statement provided by the methods described above. This method includes: determining a natural language question input by a user and a pattern item corresponding to the natural language question; inputting the natural language question and the pattern item into a conversion model to obtain a multi-model data query intermediate representation output by the conversion model; the conversion model being trained based on sample natural language questions and sample pattern items carrying a first real label; determining a target query statement template from at least two preset query statement templates based on the multi-model data query intermediate representation; and compiling the multi-model data query intermediate representation according to the target query statement template to obtain a multi-model data query statement.

[0084] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for converting natural language questions into multi-model data query statements, characterized in that, include: Determine the natural language problem in the user input; Based on the natural language question, the corresponding pattern item is obtained. The natural language question and the pattern item are input into the transformation model to obtain the multi-model data query intermediate representation output by the transformation model. The conversion model is trained based on sample natural language questions and sample pattern terms carrying a first real label; wherein, obtaining the corresponding pattern terms based on the natural language question includes: determining an initial embedding representation based on the pattern terms of the natural language question and multi-model data patterns; inputting the initial embedding representation into a graph neural network to obtain candidate pattern terms output by the graph neural network and the occurrence probability of the candidate pattern terms; the graph neural network is trained based on the initial embedding representation of samples carrying a second real label; and determining the pattern term corresponding to the natural language question from the candidate pattern terms based on a set probability range. Based on the multi-model data query intermediate representation, the target query statement template is determined from at least two preset query statement templates; The multi-model data query statement is obtained by compiling the intermediate representation of the multi-model data query based on the target query statement template.

2. The method for converting natural language questions into multi-model data query statements according to claim 1, characterized in that, Prior to training based on the natural language question carrying the first real label and the pattern term, the method further includes: The query statement template is extracted from the multi-model data query statement corresponding to the sample natural language question. The first real label is obtained by forward compilation based on the query statement template and the sample multi-model data query statement; the first real label is the intermediate representation of the multi-model data query corresponding to the sample multi-model data query statement.

3. The method for converting natural language questions into multi-model data query statements according to claim 1, characterized in that, Before compiling the intermediate representation of the multi-model data query according to the target query statement template to obtain the multi-model data query statement, the method further includes: The intermediate representation of the multi-model data query is corrected and detected based on at least one of the correction rules: keyword correction, pattern correction, sub-statement rearrangement, and symbol matching.

4. The method for converting natural language questions into multi-model data query statements according to claim 1, characterized in that, Prior to training based on sample natural language questions and sample pattern terms carrying the first true label, the method further includes: Obtain a structured query language dataset; the structured query language dataset includes structured data and a first statement pair, wherein the first statement pair is a natural language question and a structured query statement pair; The structured data is converted into at least one type of unstructured data, and multi-model data is obtained based on the structured data and the unstructured data; the first statement pair is rewritten into a natural language question and target multi-model data query statement pair to obtain a multi-model query language dataset. The sample natural language question is determined based on the natural language question in the multi-model query language dataset, and the sample pattern item is determined based on the target multi-model data query statement pair corresponding to the natural language question.

5. The method for converting natural language questions into multi-model data query statements according to claim 1, characterized in that, Training is based on sample natural language questions and sample pattern terms carrying first true labels, including: The initial embedding representation of the sample is input into the original graph neural network to obtain the candidate items output by the original graph neural network and the occurrence probability corresponding to the candidate pattern items; Determine the loss function values ​​of the candidate pattern item and the second true label, and adjust the parameters of the original graph neural network according to the loss function values; determine the direct connection relationship of each node in the graph structure data model, and adjust the occurrence probability corresponding to the candidate pattern item according to the direct connection relationship; The graph neural network is obtained when the original graph neural network is determined to be converged.

6. A device for converting natural language questions into multi-model data query statements, characterized in that, include: The input information determination module is used to determine the natural language questions input by the user; An intermediate representation generation module is used to obtain the corresponding pattern item based on the natural language question, input the natural language question and the pattern item into the transformation model, and obtain the multi-model data query intermediate representation output by the transformation model; The conversion model is trained based on sample natural language questions and sample pattern terms carrying a first real label; wherein, obtaining the corresponding pattern terms based on the natural language question includes: determining an initial embedding representation based on the pattern terms of the natural language question and multi-model data patterns; inputting the initial embedding representation into a graph neural network to obtain candidate pattern terms output by the graph neural network and the occurrence probability of the candidate pattern terms; the graph neural network is trained based on the initial embedding representation of samples carrying a second real label; and determining the pattern term corresponding to the natural language question from the candidate pattern terms based on a set probability range. The query statement template determination module is used to determine the target query statement template from at least two preset query statement templates based on the intermediate representation of the multi-model data query. The query statement generation module is used to compile and process the intermediate representation of the multi-model data query according to the target query statement template to obtain the multi-model data query statement.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method for converting natural language questions into multi-model data query statements as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for converting natural language questions into multi-model data query statements as described in any one of claims 1 to 5.

Images