Metadata-based question and answer method and related device

Abstract

The invention discloses a question answering method based on metadata and a related device, which can be used in the field of artificial intelligence, and the method comprises the following steps: obtaining an initial question input by a user; splitting the initial question into a plurality of sub-questions through a large language model; performing target identification on each sub-problem to obtain a retrieval target; performing mixed retrieval on each sub-question in combination with the initial question in a retrieval library to obtain retrieval knowledge; then, a target tool is selected from the tool set based on the retrieval target; and if the retrieval target comprises any one of the index, the field and the blood relationship and the target tool is not a large language model tool, processing the retrieval knowledge corresponding to the sub-question based on the target tool corresponding to the sub-question to obtain answer content. Therefore, the relatively complex initial question is disassembled into a plurality of sub-questions, and knowledge retrieval is performed on each sub-question, so that high-accuracy retrieval knowledge can be obtained from a complex retrieval library, and the accuracy of the answer content generated in the question and answer scene is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a metadata-based question-answering method and related devices. Background Art

[0002] With the development of information technology, technologies such as database and big data have been widely used against the backdrop of a surge in data volume and diversification of data types.

[0003] Since metadata can describe key information such as the structure, meaning, source and purpose of data, when it is necessary to quickly locate and identify the required information from a complex data environment, metadata is usually retrieved to obtain the required information. At present, metadata knowledge retrieval is usually performed based on the large language model RAG (Retrieval-Augmented Generation) to achieve knowledge question and answer. In this method, the metadata document data needs to be segmented in advance by segmentation methods such as fixed word count, natural sentence segments or semantic segmentation, and the segmented documents are vectorized and stored in the vector database; after the user enters a natural language query statement, the question is vectorized in the same way, and then the relevant segmented documents are retrieved from the vector database through the vectorization result, and the queried segment is used as the input of the large language model to generate the answer content. However, in scenarios such as the data middle platform containing multiple business wide tables, there are many wide tables, many fields, and many enumeration values, and there are complex associations between tables and fields. In the solution of knowledge retrieval based on the large language model RAG, each table and field is usually regarded as an independent block, which is difficult to handle these associations, which will lead to inaccurate generated answer content.

[0004] Therefore, how to improve the accuracy of the answers generated in question-and-answer scenarios becomes a problem that needs to be solved. Summary of the invention

[0005] Based on the above problems, the present application provides a metadata-based question-answering method and related devices, which can improve the accuracy of the answer content generated in the question-answering scenario.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a metadata-based question-answering method, the method comprising:

[0008] Get the initial question from the user;

[0009] Using a large language model, split the initial question into multiple sub-questions;

[0010] Perform target identification on each of the sub-questions to obtain the retrieval targets corresponding to each of the sub-questions; perform mixed retrieval on each of the sub-questions in combination with the initial question in the retrieval library to obtain the retrieval knowledge corresponding to each of the sub-questions; the retrieval target includes at least one of an indicator, a field, and a blood relationship;

[0011] Based on the search target, a target tool is selected from a pre-established tool set; the tool set includes at least one of a table query tool, a field query tool, an indicator query tool, a lineage query tool, an analysis method query tool, a domain knowledge query tool, and a large language model tool;

[0012] If the search target includes any one of an indicator, a field, and a blood relationship, and the target tool is not a large language model tool, the search knowledge corresponding to the sub-question is processed based on the target tool corresponding to the sub-question to obtain the answer content.

[0013] Optionally, the performing mixed search on each of the sub-questions in combination with the initial question in a search library to obtain search knowledge corresponding to each of the sub-questions includes:

[0014] Perform entity recognition on each of the sub-questions, and extract entities corresponding to each of the sub-questions;

[0015] Rewrite the query based on the entities corresponding to each of the sub-questions, and convert the sub-questions into retrieval questions;

[0016] Based on each search question and the initial question, a hybrid search including text segmentation search and text vector search is performed in the search library to obtain the search knowledge corresponding to each of the sub-questions.

[0017] Optionally, based on each search question and the initial question, a mixed search including text segmentation search and text vector search is performed in the search library to obtain search knowledge corresponding to each of the sub-questions, and the method further includes:

[0018] Based on the re-ranking technology, the search knowledge is scored to obtain a search score;

[0019] If there is a target sub-question whose retrieval scores of all retrieved knowledge are less than a preset threshold, a prompt answer indicating that no knowledge related to the target sub-question has been retrieved is output.

[0020] Optionally, after selecting a target tool from a pre-established tool set based on the search target, the method further comprises:

[0021] If the number of target tools corresponding to the sub-question is greater than the preset number, a prompt answer suggesting multiple rounds of dialogue is output.

[0022] Optionally, after selecting a target tool from a pre-established tool set based on the search target, the method further comprises:

[0023] If the search target does not include any of the indicators, fields and blood relations, or the target tool is a large language model tool, the answer content based on the search knowledge is generated by the large language model.

[0024] Optionally, after obtaining the initial question input by the user, the method further includes:

[0025] If it is determined through the classification model or the large language model that the initial question does not belong to a metadata question, a prompt answer indicating that the question cannot be answered is output.

[0026] Optionally, before obtaining the initial question, the method further includes:

[0027] The metadata, analysis method data and domain knowledge data are structured and stored to obtain second structured data; the metadata includes table data, field data and indicator data;

[0028] Analyze the blood relationship between table data, between table data and field data, between field data, between indicator data, and between indicator data and field data in the metadata;

[0029] The blood relationship obtained through analysis is stored in a graph database based on a graph structure to obtain first structured data; the second structured data and the first structured data constitute a retrieval library.

[0030] In a second aspect, an embodiment of the present application provides a metadata-based question-answering device, the device comprising: an acquisition module, a splitting module, an identification module, a retrieval module, a selection module, and an answering module;

[0031] The acquisition module is used to acquire the initial question input by the user;

[0032] The splitting module is used to split the initial question into multiple sub-questions through a large language model;

[0033] The identification module is used to perform target identification on each of the sub-questions to obtain the search targets corresponding to each of the sub-questions; the search targets include at least one of an indicator, a field and a blood relationship;

[0034] The retrieval module is used to perform mixed retrieval on each of the sub-questions in combination with the initial question in the retrieval library to obtain retrieval knowledge corresponding to each of the sub-questions;

[0035] The selection module is used to select a target tool from a pre-established tool set based on the search target; the tool set includes at least one of a table query tool, a field query tool, an indicator query tool, a lineage query tool, an analysis method query tool, a domain knowledge query tool, and a large language model tool;

[0036] The answer module is used to process the retrieval knowledge corresponding to the sub-question based on the target tool corresponding to the sub-question to obtain the answer content when the retrieval target includes any one of an indicator, a field and a blood relationship and the target tool is not a large language model tool.

[0037] In a third aspect, an embodiment of the present application provides a metadata-based question-answering device, the device comprising: a memory and a processor;

[0038] The memory is used to store program code and transmit the program code to the processor;

[0039] The processor is used to execute the steps of the metadata-based question-answering method described in any implementation manner of the first aspect according to the program code.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a metadata-based question-and-answer device, the metadata-based question-and-answer device performs the steps of the metadata-based question-and-answer method described in any implementation method in the first aspect.

[0041] Compared with the prior art, this application has the following beneficial effects:

[0042] The embodiment of the present application provides a metadata-based question-answering method, in which, first, the initial question input by the user is obtained; then, the initial question is split into multiple sub-questions through a large language model; then, target identification is performed on each of the sub-questions to obtain the retrieval targets corresponding to each of the sub-questions; each of the sub-questions is combined with the initial question to perform a mixed search in the retrieval library to obtain the retrieval knowledge corresponding to each of the sub-questions; then, based on the retrieval target, a target tool is selected from a pre-established tool set; finally, if the retrieval target includes any one of an indicator, a field, and a blood relationship, and the target tool is not a large language model tool, the retrieval knowledge corresponding to the sub-question is processed based on the target tool corresponding to the sub-question to obtain the answer content. Thus, the more complex initial question is decomposed into multiple sub-questions, and knowledge retrieval is performed for each sub-question, so that high-accuracy retrieval knowledge can be obtained from the complex retrieval library; then, based on the retrieval target, a suitable target tool is selected to process the high-accuracy retrieval knowledge, so that each sub-question can be flexibly answered in a suitable form, thereby improving the accuracy of the answer content generated in the question-answering scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A flowchart of a metadata-based question-answering method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a search library construction process provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a metadata-based question-answering device provided in an embodiment of the present application;

[0047] Figure 4 A structural diagram of a metadata-based question-and-answer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The metadata-based question-answering method and related devices provided in the present application can be used in the field of artificial intelligence. The above is only an example and does not limit the application field of the metadata-based question-answering method and related devices provided in the present application.

[0049] The terms "first", "second", "third" and "fourth" etc. in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0050] In the embodiments of the present application, the words "as an example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "as an example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the words "as an example" or "for example" is intended to present the relevant concepts in a specific way.

[0051] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] See also Figure 1 , which is a flow chart of a metadata-based question-answering method provided in an embodiment of the present application, the method comprising:

[0054] S101: Obtaining an initial question input by a user.

[0055] Specifically, the initial question is a question that the user wants to ask input through the question-and-answer window.

[0056] Optionally, after obtaining the initial question, the initial question can be classified by a classification model, or a large language model can be used to perform a question-and-answer session on whether the initial question is a metadata question, thereby determining whether the initial question is a metadata question. If the initial question is a metadata question, the subsequent steps can be performed; if the initial question is not a metadata question, a prompt answer indicating that the question cannot be answered can be output to end the question-and-answer session. For example, the output can be “Sorry, I can only answer questions related to metadata and data analysis, and cannot answer other questions. Please ask again.” The metadata question is a question related to at least one of metadata, data analysis, and domain knowledge.

[0057] S102: Split the initial question into multiple sub-questions using a large language model.

[0058] As an example, the initial question can be split into multiple independent minimum sub-questions through the large language model question-answering method, so that each sub-question can be answered separately later.

[0059] When faced with complex initial questions, the large language model can be used to answer questions and split the initial question into multiple independent minimum sub-questions, making it easier to analyze and answer each sub-question separately. In this way, it can ensure that each sub-question is fully understood and answered in detail, thereby improving the accuracy of the overall answer content.

[0060] S103: Perform target recognition on each sub-question respectively to obtain the retrieval target corresponding to each sub-question.

[0061] The search target includes at least one of an index, a field, and a blood relationship. In addition, the search target may also include, but is not limited to, table data, analysis methods, domain knowledge, etc. Optionally, the search target may be a single target or a composite target that includes multiple targets at the same time.

[0062] As an example, target recognition can be performed through a classification model to determine the content that needs to be answered for each sub-question, thereby obtaining the retrieval target of each sub-question.

[0063] S104: Perform mixed search on each sub-question in combination with the initial question in the search database to obtain search knowledge corresponding to each sub-question.

[0064] As an example, we can first perform entity recognition on each sub-question and extract the entities corresponding to each sub-question; then, perform query rewriting based on the entities corresponding to each sub-question and convert the sub-question into a retrieval question; finally, based on each retrieval question and the initial question, perform a hybrid search including text segmentation retrieval and text vector retrieval in the retrieval library to obtain the retrieval knowledge corresponding to each sub-question.

[0065] Specifically, we can first perform entity recognition on each sub-question through natural language processing tools such as spaCy, NLTK, BERT, etc., extract entities with specific meanings from the sub-questions, such as product names, locations, or customer names, and obtain the entities corresponding to each sub-question. For example, if the sub-question is "What are the source fields of new insurance premiums", we can extract "new insurance premiums" and "source fields" through entity recognition, which are the two entities corresponding to the sub-question "What are the source fields of new insurance premiums".

[0066] Then, based on the entities corresponding to each sub-question, query rewriting can be performed to convert the sub-questions into search questions that are more suitable for the search system to understand. For example, based on the extracted entities, the sub-question "What are the source fields of new insurance premiums" can be rewritten as "Find the source fields of new insurance premiums", thereby converting the sub-questions expressed in natural language into more precise search questions and improving the accuracy of the query.

[0067] Finally, these search questions and the initial questions are combined to perform hybrid search using the search library. Hybrid search includes token-based retrieval and vector-based retrieval. Token-based retrieval can be implemented through search engines such as Elasticsearch or Solr. During the retrieval process, the sub-questions are decomposed into individual words, and then the matching items of these words are found in the search library based on word frequency and reverse order. Vector-based retrieval can be implemented through models such as Sentence Transformers. During the retrieval process, the sub-questions are converted into vector form, and the retrieval results are obtained by calculating the similarity between the sub-question vector and the vectorized metadata in the search library. Hybrid retrieval, which combines text analysis retrieval and text vector retrieval, can cover all levels of the problem more comprehensively, thereby providing accurate retrieval knowledge for each sub-question.

[0068] Optionally, after obtaining the retrieval knowledge corresponding to each sub-question, the retrieval knowledge can be scored based on the re-ranking technology to obtain a retrieval score; if there is a target sub-question whose retrieval scores of all retrieval knowledge are less than a preset threshold, a prompt answer indicating that the relevant knowledge of the target sub-question has not been retrieved is output.

[0069] As an example, in the process of scoring the retrieval knowledge based on the re-ranking technology, multiple factors such as keyword matching and content relevance can be comprehensively considered, so that the retrieval score can be used to reflect the relevance and accuracy of the retrieval knowledge to the sub-question. For a sub-question, multiple retrieval knowledge may be retrieved. The retrieval knowledge can be filtered by combining the retrieval score and the preset threshold. For example, if the preset threshold is 0.3, after obtaining the retrieval score of the retrieval knowledge, the retrieval knowledge with a retrieval score lower than 0.3 is filtered out; if there is a target sub-question in which the retrieval scores of all retrieval knowledge are less than 0.3, a prompt answer can be output to indicate that the relevant knowledge of the target sub-question has not been retrieved, for example, "When answering the minimum sub-question x, no relevant indicators, table knowledge or field knowledge information was retrieved. Please ask again.", and the question and answer is ended. In this way, the retrieval knowledge with low relevance to the sub-question can be removed, and the accuracy of the retrieval knowledge used to answer the sub-question can be further improved.

[0070] S105: Based on the search target, select a target tool from a pre-established tool set.

[0071] Specifically, the tool set may include at least one of the following: table query tool, field query tool, indicator query tool, lineage query tool, analysis method query tool, domain knowledge query tool, and large language model tool. Among them, the table query tool can query table name, attribute, purpose, owner and other information through given information; the field query tool can query field name, description and other information through given field or additional table name; the indicator query tool can query indicator name, purpose, attribute and other information through given name; the lineage query tool can query table, field and indicator lineage information through given type and name; the analysis method query tool can query analysis method purpose, dimension and operation step information through given analysis method name; the domain knowledge query tool can query domain knowledge through given questions. The difference between the domain knowledge query tool and the table query tool, field query tool, indicator query tool and lineage query tool is that the large language model can be called to summarize domain knowledge and generate answer content; the large language model tool can summarize and extract text from the search knowledge through the large language model to generate answer content.

[0072] As an example, based on the retrieval target, the planning capability of the large language model can be used to plan the sub-problems, and one or more target tools can be selected from the pre-established tool set. Specifically, a single operation step, a two-step operation step, and a multi-step operation step can be planned based on the retrieval target, where a single operation step corresponds to a retrieval target that can be completed by one target tool, such as querying table information, field information, or lineage information, etc.; a two-step operation step corresponds to a retrieval target that can only be completed by two target tools (i.e., a composite target), for example, it is necessary to combine table information and understand which fields are used in the table through analytical methods, etc.; similar to the two-step operation step, a multi-step operation step corresponds to a retrieval target that requires more target work to complete.

[0073] Optionally, the maximum number of target tools can be set. If the number of target tools corresponding to the sub-questions is greater than the preset number, a prompt answer suggesting multiple rounds of dialogue can be output. For example, the preset number can be 2. When the number of target tools corresponding to the sub-questions is greater than 2, a prompt answer such as "This question has exceeded two steps of tool planning, and may not be able to accurately answer the question you asked. It is recommended that you ask step by step through multiple rounds of dialogue" can be output to end the question and answer. In this way, the complexity of question and answer system planning can be reduced and the accuracy of the answer content can be improved.

[0074] S106: Determine whether the search target includes any of the indicators, fields and blood relations. If so, execute step S107; if not, execute step S109.

[0075] S107: Determine whether the target tool is a large language model tool, if so, execute step S109; if not, execute step S108.

[0076] S108: Processing the search knowledge corresponding to the sub-question based on the target tool corresponding to the sub-question to obtain the answer content.

[0077] Specifically, for each sub-question where the retrieval target includes any one of indicators, fields, and kinship relations, and the target tool is not a large language model, the corresponding target tool can be used to process the retrieval knowledge. For example, if the target tool is a table query tool, the retrieval knowledge can be processed by the table query tool, and the table entry information in the retrieval knowledge can be selected as the answer content of the corresponding sub-question. In this way, a variety of metadata formats can be used more effectively.

[0078] Optionally, while retaining the format of the answers to each sub-question, the large language model can be used to combine the answers to each sub-question to generate an answer to the initial question, or the answers to each sub-question can be concatenated to obtain an answer to the initial question.

[0079] S109: Generate answer content based on the retrieved knowledge through a large language model.

[0080] Specifically, for sub-questions where the search target does not include any of the indicators, fields, and blood relationships, or the target tool is a large language model, the answer content can be directly generated based on the corresponding search knowledge through the large language model.

[0081] Therefore, in an embodiment of the present application, a relatively complex initial question is broken down into multiple sub-questions, and knowledge retrieval is performed on each sub-question separately, so that highly accurate retrieval knowledge can be obtained from a complex retrieval library; then, based on the retrieval target, a suitable target tool is selected to process the highly accurate retrieval knowledge, so that each sub-question can be flexibly answered in a suitable form, thereby improving the accuracy of the answer content generated in the question-and-answer scenario.

[0082] In an embodiment of the present application, the data in the retrieval library is pre-stored in a structured form through offline processing, and the retrieval library includes metadata, analysis method data, and domain knowledge data, wherein the metadata may include table data, field data, and indicator data.

[0083] Optionally, for the construction of a retrieval library, the blood relationship between table data, between table data and field data, between field data, between indicator data, and between indicator data and field data can be analyzed; the blood relationship obtained by the analysis is stored in a graph database based on a graph structure to obtain first structured data; metadata, analysis method data, domain knowledge data, and blood relationship are structured and stored to obtain second structured data; the second structured data and the first structured data constitute a retrieval library.

[0084] Specifically, see Figure 2 , which is a schematic diagram of a retrieval library construction process provided in an embodiment of the present application.

[0085] Table data refers to the content related to table creation, which records the table description information. For example, it may include table name, project space, location, department, field name, field type, field description and other data. The processing process of table data can be recorded through table processing SQL statements. Through table processing SQL statements, the blood relationship between table data, between table data and field data, and between field data can be obtained.

[0086] For table data and field data, DDL (Data Definition Language) structured processing can be performed table by table and SQL statement by table, where the table is used to process table, field and other information, and the SQL statement is used to parse and process the lineage information related to the table and field.

[0087] As an example, the fields can be classified and clustered using traditional machine learning clustering methods, deep learning vectorized clustering, or large model-based question-answer generation, and the clustered categories can be stored in the field structured information, so that the same category information can be used to search for content expansion during the question-answering process. Since the input length of the text block vectorization is much smaller than the entire table structure information length, the fields can be grouped into one or more groups by combining clustering labels, table structures, and text vectorization input length limits, and all field information can be recursively summarized, and the summarized content is used as the extended information of the table description to form an information summary. Then, based on the information summary, the table description including the extended information and the grouped field information are vectorized, and then combined with the original structured information of the blocks, such as table name, field name, blood dependency, proper nouns, etc., the vectorized information of each block is structured and stored to form a component of the second structured data.

[0088] In order to perform more comprehensive information retrieval and recall during the question-answering process, the table data and field data can be further expanded to provide extended information. For example, the proper names in the table data and field data can be extracted through entities, and the non-standard descriptions in the table data or field data can be changed to more standardized descriptions through description rewriting; the 1st to 2nd layer of lineage information can be stored in the structural information of the table or field, and the lineage information between the data can be indicated by marking bits or names, so that the tool can perform lineage calls and queries based on this information.

[0089] The blood relationship information between data can be stored in a structured manner in the form of a graph based on a graph structure, for example, it can be stored in a graph database to form a component of the first structured data. Thus, the blood relationship of different information can be queried through tools, for example, querying multi-layer blood relationships according to depth or querying blood relationships according to the source and destination nodes of the graph.

[0090] For indicator data, information related to the indicator data can be collected from systems such as the indicator center, such as name, definition, dimension information, owner information, derived information, and other data. Since indicators are obtained by combining and calculating data such as fields or other indicators, they implicitly have blood relationships. By analyzing these blood relationships, we can obtain the blood relationships between indicator data and between indicator data and field data. These blood relationships are stored in the graph database in the form of graph relationships, which means that another component of the first structured data is obtained. As a result, it becomes possible to answer complex questions through indicators. For example, you can understand which indicators are applicable through fields or understand their derived indicators through indicators. Indicator information can be stored in a structured format such as JSON.

[0091] Optionally, entities and proper nouns can be extracted from the indicator name to rewrite the indicator description in a standardized manner, such as changing spoken language into written language through a language model to expand the indicator data. In this way, the indicator data can be expanded to a more forward description, so that more comprehensive indicator information can be retrieved during the question-answering process. The expanded indicator data is vectorized, and the information is grouped and vectorized if the data is too long. The vectorized results are stored in a structured manner to form a component of the second structured data.

[0092] The analysis method data includes the basic descriptive information such as the dimensions and indicators required for the analysis method. The analysis method is disassembled according to the attributes such as name, description, purpose, required dimensions and indicators, and calculation step description, and the disassembled attributes are stored in a structured manner, for example, in JSON key-value pairs; then, the structured information is vectorized, and the vectorized results and structured information are stored in a structured manner to form another component of the second structured data to support the retrieval of text content and vectorized methods.

[0093] For domain knowledge data, knowledge data preprocessing can be performed in the manner of existing general large model knowledge documents, that is, data segmentation, vectorization after segmentation, and structured information storage after vectorization to form another component of the second structured data.

[0094] In the embodiment of the present application, richer knowledge information such as entity information and blood relationship is added to the search library through expansion.

[0095] See also Figure 3 , this figure is a schematic diagram of a metadata-based question-answering device provided in an embodiment of the present application, and the device includes: an acquisition module 301, a splitting module 302, an identification module 303, a retrieval module 304, a selection module 305 and an answering module 306.

[0096] An acquisition module 301 is used to acquire an initial question input by a user;

[0097] A splitting module 302 is used to split the initial question into multiple sub-questions through a large language model;

[0098] The identification module 303 is used to identify the target of each sub-question and obtain the search target corresponding to each sub-question; the search target includes at least one of an indicator, a field and a blood relationship;

[0099] Retrieval module 304, used to perform mixed retrieval on each sub-question in combination with the initial question in the retrieval library to obtain retrieval knowledge corresponding to each sub-question;

[0100] A selection module 305 is used to select a target tool from a pre-established tool set based on the search target; the tool set includes at least one of a table query tool, a field query tool, an indicator query tool, a lineage query tool, an analysis method query tool, a domain knowledge query tool, and a large language model tool;

[0101] The answer module 306 is used to process the search knowledge corresponding to the sub-question based on the target tool corresponding to the sub-question to obtain the answer content when the search target includes any one of indicators, fields and blood relations and the target tool is not a large language model tool.

[0102] Therefore, in an embodiment of the present application, a relatively complex initial question is broken down into multiple sub-questions, and knowledge retrieval is performed on each sub-question separately, so that highly accurate retrieval knowledge can be obtained from a complex retrieval library; then, based on the retrieval target, a suitable target tool is selected to process the highly accurate retrieval knowledge, so that each sub-question can be flexibly answered in a suitable form, thereby improving the accuracy of the answer content generated in the question-and-answer scenario.

[0103] Optionally, the retrieval module 304 includes an extraction unit, a rewriting unit and a retrieval unit; wherein the extraction unit is used to perform entity recognition on each sub-question respectively, and extract the entities corresponding to each sub-question respectively; the rewriting unit is used to perform query rewriting based on the entities corresponding to each sub-question respectively, and convert the sub-question into a retrieval question; the retrieval unit is used to perform a hybrid search including text segmentation retrieval and text vector retrieval in the retrieval library based on each retrieval question and the initial question, and obtain the retrieval knowledge corresponding to each sub-question respectively.

[0104] Optionally, another metadata-based question-and-answer device provided by the present application also includes: a scoring module and a first prompt module; wherein the scoring module is used to score the retrieved knowledge based on the reranking technology to obtain a retrieval score; the first prompt module is used to output a prompt answer indicating that the knowledge related to the target sub-question has not been retrieved when there is a target sub-question for which the retrieval scores of all retrieved knowledge are less than a preset threshold.

[0105] Optionally, another metadata-based question-and-answer device provided by the present application also includes: a second prompt module, which is used to output a prompt answer suggesting multiple rounds of dialogue when the number of target tools corresponding to the sub-question is greater than a preset number.

[0106] Optionally, the answer module 306 is also used to generate answer content based on the search knowledge through the large language model when the search target does not include any of the indicators, fields and blood relations, or the target tool is a large language model tool.

[0107] Optionally, another metadata-based question-answering device provided by the present application also includes: a third prompt module, which is used to output a prompt answer representing that the question cannot be answered when the over-classification model or the large language model determines that the initial question does not belong to the metadata question.

[0108] Optionally, another metadata-based question-and-answer device provided by the present application also includes: a retrieval library construction module, which is used to structuredly store metadata, analysis method data and domain knowledge data to obtain second structured data; metadata includes table data, field data and indicator data; analyzing the blood relationship between table data, between table data and field data, between field data, between indicator data, and between indicator data and field data in the metadata; storing the blood relationship obtained by the analysis in a graph database based on a graph structure to obtain first structured data; the second structured data and the first structured data constitute a retrieval library.

[0109] See also Figure 4 , this figure is a structural diagram of a metadata-based question-answering device provided in an embodiment of the present application, and the device includes: a memory 401 and a processor 402.

[0110] Memory 401: used to store program codes and transmit the program codes to the processor.

[0111] Processor 402: used to execute the steps of the metadata-based question-answering method according to the instructions in the program code.

[0112] In addition, the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a metadata-based question-and-answer device, the metadata-based question-and-answer device performs the steps of the above-mentioned metadata-based question-and-answer method.

[0113] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A metadata-based question-answering method, characterized in that: The method comprises: Get the initial question from the user; Using a large language model, split the initial question into multiple sub-questions; Perform target identification on each of the sub-questions to obtain the retrieval targets corresponding to each of the sub-questions; perform mixed retrieval on each of the sub-questions in combination with the initial question in the retrieval library to obtain the retrieval knowledge corresponding to each of the sub-questions; the retrieval target includes at least one of an indicator, a field, and a blood relationship; Based on the search target, a target tool is selected from a pre-established tool set; the tool set includes at least one of a table query tool, a field query tool, an indicator query tool, a lineage query tool, an analysis method query tool, a domain knowledge query tool, and a large language model tool; If the search target includes any one of an indicator, a field, and a blood relationship, and the target tool is not a large language model tool, the search knowledge corresponding to the sub-question is processed based on the target tool corresponding to the sub-question to obtain the answer content.

2. The method according to claim 1, characterized in that The hybrid search is performed on each of the sub-questions in combination with the initial question in the search library to obtain the search knowledge corresponding to each of the sub-questions, including: Perform entity recognition on each of the sub-questions, and extract entities corresponding to each of the sub-questions; Rewrite the query based on the entities corresponding to each of the sub-questions, and convert the sub-questions into retrieval questions; Based on each search question and the initial question, a hybrid search including text segmentation search and text vector search is performed in the search library to obtain the search knowledge corresponding to each of the sub-questions.

3. The method according to claim 2, characterized in that Based on each search question and the initial question, a hybrid search including text segmentation search and text vector search is performed in the search library to obtain search knowledge corresponding to each sub-question, the method further includes: Based on the re-ranking technology, the search knowledge is scored to obtain a search score; If there is a target sub-question whose retrieval scores of all retrieved knowledge are less than a preset threshold, a prompt answer indicating that no knowledge related to the target sub-question has been retrieved is output.

4. The method according to claim 1, characterized in that: After selecting a target tool from a pre-established tool set based on the search target, the method further includes: If the number of target tools corresponding to the sub-question is greater than the preset number, a prompt answer suggesting multiple rounds of dialogue is output.

5. The method according to claim 1, characterized in that After selecting a target tool from a pre-established tool set based on the search target, the method further includes: If the search target does not include any of the indicators, fields and blood relations, or the target tool is a large language model tool, the answer content based on the search knowledge is generated by the large language model.

6. The method according to claim 1, characterized in that After obtaining the initial question input by the user, the method further includes: If it is determined through the classification model or the large language model that the initial question does not belong to a metadata question, a prompt answer indicating that the question cannot be answered is output.

7. The method according to claim 1, characterized in that Before obtaining the initial question, the method further includes: Analyze the blood relationship between table data, between table data and field data, between field data, between indicator data, and between indicator data and field data in the metadata; the metadata includes table data, field data, and indicator data; The blood relationship obtained by the analysis is stored in a graph database based on the graph structure to obtain first structured data; The metadata, analysis method data, domain knowledge data and the blood relationship are structured and stored to obtain second structured data; the second structured data and the first structured data constitute a retrieval library.

8. A metadata-based question-answering device, characterized in that: The device comprises: an acquisition module, a splitting module, an identification module, a retrieval module, a selection module and an answering module; The acquisition module is used to acquire the initial question input by the user; The splitting module is used to split the initial question into multiple sub-questions through a large language model; The identification module is used to perform target identification on each of the sub-questions to obtain the search targets corresponding to each of the sub-questions; the search targets include at least one of an indicator, a field and a blood relationship; The retrieval module is used to perform mixed retrieval on each of the sub-questions in combination with the initial question in the retrieval library to obtain retrieval knowledge corresponding to each of the sub-questions; The selection module is used to select a target tool from a pre-established tool set based on the search target; the tool set includes at least one of a table query tool, a field query tool, an indicator query tool, a lineage query tool, an analysis method query tool, a domain knowledge query tool, and a large language model tool; The answer module is used to process the retrieval knowledge corresponding to the sub-question based on the target tool corresponding to the sub-question to obtain the answer content when the retrieval target includes any one of an indicator, a field and a blood relationship and the target tool is not a large language model tool.

9. A metadata-based question-answering device, characterized in that: The device comprises: a memory and a processor; The memory is used to store program codes and transmit the program codes to the processor; The processor is configured to execute the steps of the metadata-based question-answering method according to any one of claims 1 to 7 according to the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program runs on the metadata-based question-answering device, the metadata-based question-answering device performs the steps of the metadata-based question-answering method according to any one of claims 1 to 7.

Images