Intelligent question answering processing method, apparatus, device, and program product

Abstract

The present disclosure provides an intelligent question and answer processing method, device, equipment and program product. The present disclosure decomposes the user question into a first task through a large-scale language model, queries a vector database according to the first task to obtain at least one candidate query statement matched with the first task, and generates a target query statement according to the at least one candidate query statement. Then, a data set corresponding to the target query statement is obtained from a business database. If the data set cannot be used to generate reply information corresponding to the user question, at least one second index having an influence relationship with a first index in the user question or the first task is obtained from a graph database according to the first index, and the at least one second index is further decomposed into a second task. The second task is continuously executed until the reply information corresponding to the user question is generated. Thus, comprehensive and accurate reply information is provided for the user.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to intelligent question-and-answer processing methods, apparatus, devices, and program products. Background Technology

[0002] With the continuous development of computer technology, many intelligent question-answering systems have emerged. For example, when a user enters a question in the user interface, the intelligent question-answering system converts the question into a Structured Query Language (SQL) statement and queries the database based on the SQL statement to obtain the data needed to answer the user's question.

[0003] However, current intelligent question-answering systems cannot provide comprehensive and accurate data analysis results, resulting in the incomplete and inaccurate response information provided to users. Summary of the Invention

[0004] According to one aspect of this disclosure, an intelligent question-answering processing method is provided, the method comprising:

[0005] The user's problem is decomposed into a task using a large-scale language model to obtain the first task;

[0006] Based on the first task, query the vector database to obtain at least one candidate query statement that matches the first task, and generate the target query statement based on the at least one candidate query statement.

[0007] Retrieve the dataset corresponding to the target query statement from the business database;

[0008] If the answer information corresponding to the user question cannot be generated based on the dataset, then at least one second indicator that is related to the first indicator is obtained from the graph database based on the first indicator in the first task or the user question.

[0009] The at least one second indicator is decomposed into a second task, and the second task is executed until the answer information corresponding to the user question is generated.

[0010] According to another aspect of this disclosure, an intelligent question-answering processing apparatus is provided, the apparatus comprising:

[0011] The task decomposition module is used to decompose the user's problem into tasks using a large-scale language model to obtain the first task.

[0012] The query module is used to query the vector database according to the first task to obtain at least one candidate query statement that matches the first task.

[0013] The generation module is used to generate a target query statement based on the at least one candidate query statement;

[0014] The first acquisition module is used to acquire the dataset corresponding to the target query statement from the business database;

[0015] The second acquisition module is used to acquire at least one second indicator that is related to the first indicator from the graph database when the answer information corresponding to the user question cannot be generated based on the dataset, based on the first indicator in the first task or the user question.

[0016] The task decomposition module is also used to decompose the at least one second indicator into a second task.

[0017] The query module, the generation module, the first acquisition module, and the second acquisition module are further configured to continue executing the second task until the answer information corresponding to the user question is generated.

[0018] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0019] Processor; and

[0020] Stored program memory,

[0021] The program includes instructions that, when executed by the processor, cause the processor to perform the method according to the first aspect.

[0022] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the method according to the first aspect.

[0023] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0024] One or more technical solutions provided in the embodiments of this application can decompose user questions step by step, accurately understand user intentions, and obtain datasets related to user questions step by step, so that the datasets obtained step by step have upstream and downstream causal logic, thereby providing users with comprehensive and accurate answer information based on these datasets with causal logic. Attached Figure Description

[0025] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0026] Figure 1A schematic diagram illustrating an application scenario to which the intelligent question-answering processing method provided in this embodiment of the disclosure is applicable;

[0027] Figure 2 A schematic diagram illustrating the construction process of the vector database and graph database provided in the embodiments of this disclosure;

[0028] Figure 3 A schematic diagram of the intelligent question-answering processing method provided in the embodiments of this disclosure;

[0029] Figure 4 A schematic diagram of the intelligent question-answering processing method provided in the embodiments of this disclosure;

[0030] Figure 5 A schematic diagram of the intelligent question-answering processing method provided in the embodiments of this disclosure;

[0031] Figure 6 This is a schematic diagram of the structure of the intelligent question-answering processing device provided in the embodiments of this disclosure;

[0032] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0033] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0034] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0035] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0036] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0037] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0038] Typically, users input their questions into the user interface, and the intelligent question-answering system converts these questions into Structured Query Language (SQL) statements. It then queries the database based on these SQL statements to obtain the data needed to answer the user's question. However, current intelligent question-answering systems cannot provide comprehensive and accurate data analysis results, leading to incomplete and inaccurate responses provided to users. To address this issue, this disclosure provides an intelligent question-answering processing method, which is described below with reference to specific embodiments.

[0039] The intelligent question-answering processing method provided in this disclosure is executed by an intelligent question-answering processing device. This device can be implemented in software and / or hardware and can be configured in an electronic device, such as a server or server cluster. A server cluster can aggregate multiple servers to provide the same service, appearing to the client as a single server. A server cluster can utilize multiple computers for parallel computing to achieve higher computing speeds, or it can use multiple computers for backup, ensuring the entire system continues to operate normally even if any machine fails. Assuming the intelligent question-answering processing device is configured in… Figure 1 In the server 11 shown, the intelligent question-answering processing method provided in this embodiment of the disclosure is... Figure 1The server 11 shown in the diagram can receive user questions (also called user query questions) sent by terminal 12. These user questions are in natural language, such as "Why did the number of daily active users decrease on April 8th?". Server 11 performs a series of analyses and processes on the user question, generates corresponding answer information, and feeds this answer information back to terminal 12. Terminal 12 displays the answer information on the user interface for the user to view. Terminal 12 can be a desktop computer, tablet computer, mobile phone, etc. During the analysis and processing of the user question by server 11, queries to the vector database and graph database are involved. It is understood that in this embodiment, the vector database and graph database are pre-built; that is, the process of building the vector database and graph database can be performed before server 11 receives the user question. Specifically, the construction process of the vector database and graph database can be performed by server 11 or by other servers. Assuming that the construction process of the vector database and graph database is performed by server 11, the construction process of the vector database and graph database will be described below with reference to specific embodiments.

[0040] Specifically, the intelligent question-answering processing method provided in this embodiment includes: constructing the vector database, wherein the vector database includes the correspondence between sample query statements and descriptive information.

[0041] For example, before building the vector database, server 11 can initialize a vector database. This initialized vector database does not contain actual data, but it has a certain initialization structure. For instance, the initialized vector database includes two columns: one called the Structured Query Language (SQL) column, and the other called the description column. The SQL column is used to store SQL statements during the construction process, and the description column is used to store the corresponding descriptive information for the SQL statements. In other words, there is a one-to-one correspondence between SQL statements and descriptive information; each SQL statement corresponds to its own descriptive information, and the descriptive information for any SQL statement can be semantic information related to that SQL statement. Furthermore, the SQL statements stored in the vector database during the construction process can be referred to as sample query statements.

[0042] This disclosure embodiment does not have format requirements for the source data used to construct the vector database. For example, this disclosure embodiment can be based on, as shown in the example below. Figure 2 The vector database is constructed using the business documents, business database, and indicator SQL query documents shown. The following section uses the business database as an example to illustrate the construction process of the vector database.

[0043] Optionally, constructing the vector database includes: obtaining at least one table from a business database; generating a sample query statement corresponding to each table in the at least one table, and obtaining metadata for each column in the table; storing the sample query statement corresponding to the table and the metadata for each column in the table into the vector database, wherein the metadata for each column in the table is used as descriptive information.

[0044] Specifically, the business database stores at least one table. For example, the business database may store multiple tables. Server 11 retrieves these multiple tables from the business database. For each of these tables, Server 11 can generate a sample query statement corresponding to that table based on the attribute information of its columns. Simultaneously, Server 11 can also extract metadata for each column in the table. For example, each column in the table corresponds to a column name (also called a field), type information (e.g., whether the column is numeric or string), and constraints (e.g., whether the column is a primary key or a foreign key). Further, Server 11 stores the sample query statement corresponding to the same table, along with the metadata of all columns included in that table, as descriptive information in the vector database. Specifically, the sample query statement is stored in the SQL column, and the descriptive information is stored in the description column. It can be understood that based on each table in the business database, a set of correspondences between sample query statements and descriptive information can be obtained. Therefore, based on multiple tables in the business database, multiple sets of correspondences between sample query statements and descriptive information can be obtained, and these multiple sets of correspondences can construct the vector database. Understandably, the constructed vector database can be stored locally on server 11 or on other servers.

[0045] Furthermore, this embodiment of the present disclosure can construct a vector database not only based on a business database, but also based on indicator SQL query documents. Specifically, server 11 can extract the SQL statements corresponding to various core business indicators from the indicator SQL query documents, as well as descriptive information such as indicator name, business meaning, calculation logic, and application scenarios. Further, the SQL statements are stored in the SQL column of the vector database, and the descriptive information is stored in the description column of the vector database.

[0046] After constructing the vector database, this embodiment can also construct a graph database, which can specifically be a business knowledge graph. The construction process of the graph database is described below with reference to specific embodiments.

[0047] Optionally, the intelligent question-answering processing method described above also includes, for example, Figure 3 The following steps are shown:

[0048] S301. Input the description information from the vector database into the large-scale language model so that the large-scale language model can extract multiple subject elements from the description information and the relationships between different subject elements. The subject elements include fields and indicators.

[0049] For example, the descriptive information from the constructed vector database can be input into a Large Language Model (LLM) for analysis and relation extraction. This allows the LLM to extract multiple subject elements from the descriptive information, as well as the relationships between different subject elements. Each subject element can be a field or indicator contained in the descriptive information. The relationships between different subject elements include influence relationships and inclusion relationships. For example, if field A is included in the calculation logic of indicator X, or indicator Y is affected by the value of field B, the relationship between field A and indicator X is an inclusion relationship (also known as a master-child relationship), and the relationship between indicator Y and field B is an influence relationship. Influence relationships can be further subdivided into direct proportional influence and inverse proportional influence.

[0050] S302. Based on the multiple subject elements in the description information and the relationships between different subject elements, construct the graph database, where each subject element is a node in the graph database, and the relationships between different subject elements are the relationships between different nodes in the graph database.

[0051] For example, a large-scale language model can be deployed on server 11. In this case, server 11 can store multiple subject elements extracted from the descriptive information by the large-scale language model, as well as the relationships between different subject elements, into a graph database, such as... Figure 2 As shown, a complete business knowledge graph is constructed, comprising multiple nodes and relationships between them. As mentioned above, a subject element can serve as a node in this business knowledge graph; that is, a node can be a field or an indicator. Furthermore, the relationships between different subject elements can be considered as the relationships between different nodes in the business knowledge graph. For example, field A and indicator X are two nodes in the business knowledge graph, and the relationship between field A and indicator X is the relationship between these two nodes. Similarly, the constructed graph database can be stored locally on server 11 or on another server.

[0052] In other embodiments, the large-scale language model can also be deployed on other servers. In this case, the large-scale language model sends multiple subject elements extracted from the description information and the relationships between different subject elements to server 11. Furthermore, server 11 stores the multiple subject elements and the relationships between different subject elements into a graph database, thereby constructing a complete business knowledge graph.

[0053] Optionally, the method further includes: acquiring structured data from the graph database, the structured data including any node and the relationships between the any node and its neighboring nodes; converting the structured data into a natural language description; and prioritizing the relationships between the any node and its neighboring nodes based on the natural language description.

[0054] For example, server 11 can retrieve structured data from a graph database. This structured data is organized by each node in the graph database; for instance, it may include any node in the graph database and the relationships between that node and its neighboring nodes. Further, server 11 outputs this structured data as a natural language description, which is then manually reviewed and corrected by relevant experts. Based on business value and reliability, the relationships are prioritized manually, such as... Figure 2 As shown. For example, any given node may have multiple neighboring nodes. If there is a relationship between any given node and each of its neighboring nodes, then there are multiple relationships between any given node and its multiple neighboring nodes. Through such manual review and correction, these multiple relationships can be prioritized. Relationships with higher priority can then be used for subsequent analysis of user issues.

[0055] Figure 4 This is a flowchart illustrating an intelligent question-answering processing method provided in an embodiment of this disclosure. This method can be executed by an intelligent question-answering processing device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a server or server cluster. For example, the intelligent question-answering processing method described in this embodiment is derived from... Figure 1 The server 11 shown executes the command. Terminal 12 can communicate with server 11. Specifically, terminal 12 displays a user interface where the user can input a question. The specific input method is not limited; it can include text input, voice input, gesture input, etc. Furthermore, terminal 12 sends the user question to server 21 so that server 21 can analyze and process the question, thereby providing a corresponding answer. The following section combines... Figure 1 A detailed introduction to intelligent question-answering processing methods, such as... Figure 4 As shown, the specific steps of this method are as follows:

[0056] S401. The user problem is decomposed into tasks using a large-scale language model to obtain the first task.

[0057] like Figure 5 The task resolution agent (ReactAgent) shown is software or hardware deployed on server 11. ReactAgent analyzes and processes user questions to provide corresponding answers. ReactAgent includes a question-answering module, a task decomposition module, a similar intent matching module, an LLM-SQL dynamic generator, an SQL query generator, a credibility judgment module, and a reconfirmation of association factors module. The LLM-SQL dynamic generator and SQL query generator can also be deployed outside of ReactAgent; that is, the LLM-SQL dynamic generator and SQL query generator are functional modules outside of ReactAgent.

[0058] Specifically, the question-answering module in ReactAgent can receive questions from terminal 12, such as user questions sent by terminal 12. Further, the question-answering module sends the user question to the task decomposition module, which can then send the user question to the large-scale language model, allowing the large-scale language model to decompose the user question into a first task. For example, if the user question is "Why did the number of daily active users decrease on April 8th?", the large-scale language model performs an initial task decomposition. During the decomposition process, the large-scale language model determines that its ultimate task is to find the reason for the decrease in daily active users on April 8th, but it first needs to query the daily active user numbers for April 8th and April 7th. Therefore, the first task is "Query the daily active user numbers for April 8th and April 7th." This first task is a task described in natural language. Further, the large-scale language model feeds this first task back to the task decomposition module. In one feasible implementation, ReactAgent can send the thought process of a large-scale language model to terminal 12. For example, ReactAgent can send text information such as "The final task is to find out the reason for the decline in daily active users on April 8, but first we need to query the daily active users on April 8 and April 7" to terminal 12, so that terminal 12 can display the text information in the user interface so that the user can understand the progress of the task through the user interface.

[0059] S402. Based on the first task, query the vector database to obtain at least one candidate query statement that matches the first task, and generate a target query statement based on the at least one candidate query statement.

[0060] For example, the task decomposition module sends the first task to the similar intent matching module, which then queries a vector database based on the first task to obtain at least one candidate query statement that matches the first task. The similar intent matching module then generates a target query statement based on this candidate query statement, for example, using a large-scale language model. Further, this large-scale language model feeds back the target query statement to the LLM-SQL dynamic generator.

[0061] S403. Obtain the dataset corresponding to the target query statement from the business database.

[0062] For example, the LLM-SQL dynamic generator sends the target query statement to the SQL query generator, which then queries the business database based on the target query statement and retrieves the dataset corresponding to that target query statement from the business database. This dataset contains the daily active user counts for April 8th and April 7th.

[0063] S404. If the answer information corresponding to the user question cannot be generated based on the dataset, then at least one second indicator that is related to the first indicator is obtained from the graph database based on the first indicator in the first task or the user question.

[0064] For example, after the SQL query engine retrieves the dataset corresponding to the target query statement from the business database, it sends the dataset to the credibility judgment module. The credibility judgment module determines whether the dataset can generate an answer to the user's question. If the dataset can generate an answer, the credibility judgment module sends "Goal Achieved" to the question-and-answer module, causing the question-and-answer module to respond to the user's question. For example, the question-and-answer module sends the response to terminal 12. If the credibility judgment module determines that the dataset cannot generate an answer, it sends "Target Anomaly" to the reconfirmation association factor module. This causes the reconfirmation association factor module to retrieve at least one second indicator related to "Daily Active Users" from the graph database, based on the first indicator in the first task or user question, such as "Daily Active Users".

[0065] For example, when the credibility assessment module receives the daily active user (DAU) counts for April 8th and April 7th, it determines that the DAU count on April 8th is lower than that on April 7th. However, it cannot determine the reason for the decrease; that is, it cannot generate the corresponding response information for the user's question based on the currently received dataset. Therefore, the credibility assessment module sends a "target anomaly" message to the reconfirmation of correlation factors module. This prompts the reconfirmation of correlation factors module to retrieve at least one second indicator that is related to the "daily active user" count from the graph database, based on the first indicator in the first task or user question, such as the "daily active user" count. For example, the reconfirmation of correlation factors module sends a query request to the graph database, which includes the "daily active user" count. The graph database queries for at least one second indicator that is related to the "daily active user" count and feeds back the at least one second indicator that is related to the "daily active user" count to the reconfirmation of correlation factors module. The at least one second indicator that is related to the "daily active user" count can be denoted as a correlation factor; that is, the graph database can feed back correlation factors to the reconfirmation of correlation factors module. For example, the graph database sends the following information back to the module for reconfirming association factors:

[0066] {"New User Percentage", "Daily Active Users", "Negative Impact"};

[0067] {"Percentage of new registrations","Number of new registrations","Positive impact"}.

[0068] In other words, the correlation factors that the graph database feeds back to the correlation factor reconfirmation module include "new addition percentage" and "new registration number".

[0069] S405. Decompose the at least one second indicator into a task to obtain a second task, and continue to execute the second task until the answer information corresponding to the user question is generated.

[0070] The reconfirmation of correlation factors module determines, based on information from the graph database, that "Daily Active Users" and "New User Ratio" are correlated, with a negative impact, while "New User Ratio" and "New Registrations" are correlated, with a positive impact. This means that although "New Registrations" has a positive impact on "New User Ratio," "New User Ratio" has a negative impact on "Daily Active Users." This can be understood as new users registering, but these new users may not remain active in subsequent days, thus negatively impacting the overall "Daily Active Users." Furthermore, the reconfirmation of correlation factors module can send "New User Ratio" and "New Registrations" to the task decomposition module. The task decomposition module can then decompose these into a second task, for example, "Query the new registrations on April 8th and April 7th." This second task is then executed within ReactAgent. It's understood that the process of executing this second task within ReactAgent is similar to the process of executing the first task within ReactAgent as described above, and will not be elaborated further here. After completing the second task, the credibility assessment module receives the number of new registrations on April 8th and April 7th. If the credibility assessment module determines that the number of new registrations on April 8th increased compared to April 7th, and the increase is consistent with the decrease in daily active users (DAU) on April 8th compared to April 7th, then the credibility assessment module can determine that the decrease in DAU on April 8th was due to the increase in new registrations. Furthermore, the credibility assessment module sends "Goal Achieved" to the question-and-answer module, prompting the module to respond to the user's question. For example, the question-and-answer module sends the response "The reason for the decrease in DAU on April 8th is the increase in new registrations" to terminal 12.

[0071] This embodiment of the disclosure decomposes user questions into tasks using a large-scale language model to obtain a first task. Based on the first task, a vector database is queried to obtain at least one candidate query statement matching the first task, and a target query statement is generated based on the at least one candidate query statement. Then, a dataset corresponding to the target query statement is obtained from a business database. If the dataset cannot generate the answer information corresponding to the user question, at least one second indicator that is related to the first indicator is obtained from a graph database based on the first task or the first indicator in the user question. The at least one second indicator is further decomposed into tasks to obtain a second task, and the second task is executed until the answer information corresponding to the user question is generated. Because this embodiment of the disclosure decomposes user questions step by step, accurately understands user intent, and obtains datasets related to user questions step by step, the datasets obtained step by step have upstream and downstream causal logic, thereby providing users with comprehensive and accurate answer information based on these datasets with causal logic.

[0072] Optionally, according to the first task, querying the vector database to obtain at least one candidate query statement matching the first task includes: sending the first task to the vector database so that the vector database matches the first task with the description information in the vector database; and determining the sample query statements corresponding to at least one description information in the vector database that matches the first task as at least one candidate query statement matching the first task.

[0073] For example Figure 5 As shown, the task decomposition module sends the first task to the similar intent matching module. The similar intent matching module sends a query request to the vector database described above, which includes the first task. Further, the vector database matches the first task with each descriptive information in the database. If the matching degree is greater than or equal to a preset value, it determines that the descriptive information matches the first task, and records the SQL statement corresponding to the descriptive information as a candidate query statement matching the first task. This allows the vector database to retrieve at least one candidate query statement matching the first task and feeds this at least one candidate query statement back to the similar intent matching module. The similar intent matching module can then generate a target query statement based on this at least one candidate query statement.

[0074] Optionally, generating a target query statement based on the at least one candidate query statement includes: inputting the at least one candidate query statement into the large-scale language model, such that the large-scale language model obtains the candidate query statement with the highest matching degree from the at least one candidate query statement; and generating the target query statement based on the limiting conditions in the first task and the candidate query statement with the highest matching degree.

[0075] For example Figure 5 As shown, the similar intent matching module can send at least one candidate query statement to the large-scale language model, allowing the model to score and rank the query statements, thereby selecting the candidate query statement with the highest matching degree to the first task. This candidate query statement with the highest matching degree can be recorded as the basic query template. Furthermore, the large-scale language model feeds back this basic query template to the similar intent matching module.

[0076] In addition, the similar intent matching module can also analyze the limiting conditions in the first task, such as the time range and filtering dimensions. For example, if the first task is to "query the number of daily active users on April 8th and April 7th", the time range is "April 8th and April 7th", and the filtering dimension is "number of daily active users", the similar intent matching module will generate the target query statement based on these limiting conditions and the candidate query statement with the highest matching degree.

[0077] Optionally, generating the target query statement based on the limiting conditions in the first task and the candidate query statement with the highest matching degree includes: inputting the limiting conditions in the first task and the candidate query statement with the highest matching degree into the large-scale language model, so that the large-scale language model updates the original content in the candidate query statement with the highest matching degree that corresponds to the limiting conditions, thereby obtaining the target query statement.

[0078] For example Figure 5As shown, the similarity intent matching module sends the constraints in the first task and the candidate query statement with the highest matching degree to the LLM-SQL dynamic generator. The LLM-SQL dynamic generator generates the target query statement based on the constraints in the first task and the candidate query statement with the highest matching degree. Specifically, the LLM-SQL dynamic generator can input the constraints in the first task and the candidate query statement with the highest matching degree into the large-scale language model. Since the candidate query statement with the highest matching degree can be used as a basic query template, and some basic information in the basic query template may be different from the constraints in the first task, for example, the time range in the basic query template is "January 1st", that is, the date pointed to by the basic query template is "January 1st", not "April 8th, April 7th" to be analyzed. Therefore, the large-scale language model can update "January 1st" in the basic query template to "April 8th, April 7th". It's understandable that this explanation uses the time range as an example to illustrate the limitations. Besides the time range, there are other limitations such as filtering dimensions or other conditions. If the filtering dimension "daily active users" in the first task differs from the original filtering dimension in the basic query template, the large-scale language model can also update the original filtering dimension in the basic query template to "daily active users" in the first task. In other words, the large-scale language model can update the basic query template based on the limitations in the first task, updating the original content of the basic query template to reflect the limitations in the first task. The updated basic query template is recorded as the target query statement. The large-scale language model then feeds this target query statement back to the LLM-SQL dynamic generator.

[0079] Furthermore, this embodiment of the disclosure does not impose format requirements on the source data used to construct the vector database. Data from multiple sources can be incorporated into the vector database, making its content more comprehensive. Therefore, by querying the vector database, more comprehensive answer information can be provided to users. By automatically constructing the vector database and graph database, accurate understanding and rapid querying of user intent are achieved. Based on user questions, business metrics are correlated and analyzed, and artificial intelligence technology is used to predict the most likely answer. Simultaneously, it can continuously learn and optimize during user interaction, continuously improving question-and-answer quality and knowledge association capabilities.

[0080] Furthermore, during the task decomposition process, the thought process of the large-scale language model is sent to the terminal, allowing users to understand the task progress through the user interface. Users can also engage in dialogue with the large-scale language model at any time during the process to guide the direction of task resolution, thereby improving the flexibility of the data question answering service.

[0081] This disclosure embodiment can also update the vector database and graph database. One update method is to generate valuable information, such as questions, keywords, SQL statements, correlation indicators, and root causes of anomalies, based on user questions and answers generated during the intelligent question-answering process. Further, the vector database and graph database are updated according to a certain strategy, thereby achieving iterative optimization of the vector database and graph database. Another update method is to manually update the vector database and graph database to remove unreliable indicator relationships from the graph database and unreliable SQL statements from the vector database. The reliability of indicator relationships or SQL statements can be measured by the following aspects: 1) frequency of occurrence, i.e., the more times they are queried or used, the higher the reliability; 2) feedback results generated in practice, such as the higher the accuracy rate when used to answer questions, the higher the reliability; 3) consistency with content in known authoritative databases, i.e., the higher the consistency, the higher the reliability; 4) the manual judgment results of relevant field experts on its reliability.

[0082] Figure 6 This is a schematic diagram of the intelligent question-answering processing device provided in an embodiment of the present disclosure. The intelligent question-answering processing device provided in this embodiment of the present disclosure can execute the processing flow provided in the method embodiment, such as... Figure 6 As shown, the intelligent question-answering processing device 60 includes: a task decomposition module 61, a query module 62, a generation module 63, a first acquisition module 64, and a second acquisition module 65. The task decomposition module 61 is used to decompose the user question into a task using a large-scale language model to obtain a first task; the query module 62 is used to query a vector database based on the first task to obtain at least one candidate query statement matching the first task; the generation module 63 is used to generate a target query statement based on the at least one candidate query statement; the first acquisition module 64 is used to obtain a dataset corresponding to the target query statement from a business database; the second acquisition module 65 is used to obtain at least one second indicator that is related to the first indicator from a graph database when the answer information corresponding to the user question cannot be generated based on the dataset, based on the first indicator in the first task or the user question; the task decomposition module 61 is also used to decompose the at least one second indicator into a task to obtain a second task; the query module 62, the generation module 63, the first acquisition module 64, and the second acquisition module 65 are also used to continue executing the second task until the answer information corresponding to the user question is generated.

[0083] Optionally, when the generation module 63 generates the target query statement based on the at least one candidate query statement, it is specifically used to: input the at least one candidate query statement into the large-scale language model, so that the large-scale language model obtains the candidate query statement with the highest matching degree from the at least one candidate query statement; and generate the target query statement based on the limiting conditions in the first task and the candidate query statement with the highest matching degree.

[0084] Optionally, when the generation module 63 generates the target query statement based on the limiting conditions in the first task and the candidate query statement with the highest matching degree, it is specifically used to: input the limiting conditions in the first task and the candidate query statement with the highest matching degree into the large-scale language model, so that the large-scale language model updates the original content in the candidate query statement with the highest matching degree that corresponds to the limiting conditions, and obtains the target query statement.

[0085] Optionally, the intelligent question-answering processing device 60 further includes: a construction module 66, used to construct the vector database, the vector database including the correspondence between sample query statements and descriptive information. When the query module 62 queries the vector database according to the first task and obtains at least one candidate query statement matching the first task, it is specifically used to: send the first task to the vector database, so that the vector database matches the first task with the descriptive information in the vector database; and determine the sample query statements corresponding to the at least one descriptive information matching the first task in the vector database as at least one candidate query statement matching the first task.

[0086] Optionally, when the construction module 66 constructs the vector database, it is specifically used to: obtain at least one table from the business database; for each table in the at least one table, generate a sample query statement corresponding to the table and obtain the metadata of each column in the table; store the sample query statement corresponding to the table and the metadata of each column in the table into the vector database, wherein the metadata of each column in the table is used as descriptive information.

[0087] Optionally, the construction module 66 is further configured to: input the description information in the vector database into the large-scale language model, so that the large-scale language model extracts multiple subject elements in the description information and the relationships between different subject elements, wherein the subject elements include fields and indicators; construct the graph database based on the multiple subject elements in the description information and the relationships between different subject elements, wherein each subject element is a node in the graph database and the relationships between different subject elements are the relationships between different nodes in the graph database.

[0088] Optionally, the intelligent question-answering processing device 60 further includes: an optimization module 67, used to acquire structured data in the graph database, the structured data including any node and the relationship between the any node and its neighboring nodes; convert the structured data into a natural language description; and assign priority to the relationship between the any node and its neighboring nodes based on the natural language description.

[0089] Figure 6 The intelligent question-answering processing device shown in the embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0090] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of this disclosure.

[0091] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to embodiments of this disclosure.

[0092] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a processor of a computer, the computer program is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0093] refer to Figure 7 The present invention describes a structural block diagram of an electronic device 70 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0094] like Figure 7As shown, the electronic device 70 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the device 70. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0095] Multiple components in electronic device 70 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 70. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 704 may include, but is not limited to, disk and optical disk. Communication unit 709 allows electronic device 70 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0096] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the methods described in the above method embodiments can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 70 via ROM 702 and / or communication unit 709. In some embodiments, the computing unit 701 can be configured by any other suitable means (e.g., by means of firmware) to perform the methods described in the above method embodiments.

[0097] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0098] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0099] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0101] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0102] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. An intelligent question-answering processing method, the method comprising: The user's problem is decomposed into a task using a large-scale language model to obtain the first task; According to the first task, a vector database is queried to obtain at least one candidate query statement that matches the first task, and a target query statement is generated based on the at least one candidate query statement; the vector database includes the correspondence between sample query statements and descriptive information; Retrieve the dataset corresponding to the target query statement from the business database; Determine whether the dataset can be used to generate a response to the user's question. If the answer information corresponding to the user question cannot be generated based on the dataset, then at least one second indicator that is related to the first indicator is obtained from the graph database based on the first indicator in the first task or the user question. The at least one second indicator is decomposed into a task to obtain a second task, and the second task is continued to be executed until the answer information corresponding to the user question is generated. Constructing the vector database includes: Retrieve at least one table from the business database; For each of the at least one tables, generate a sample query statement corresponding to the table and obtain the metadata of each column in the table; The sample query statement corresponding to the table and the metadata of each column in the table are stored in the vector database, and the metadata of each column in the table is used as descriptive information. The method further includes: The descriptive information from the vector database is input into the large-scale language model so that the large-scale language model can extract multiple subject elements from the descriptive information and the relationships between different subject elements. The subject elements include fields and indicators. Based on the multiple subject elements in the description information and the relationships between different subject elements, the graph database is constructed. Each subject element is a node in the graph database, and the relationships between different subject elements are the relationships between different nodes in the graph database. The relationships include influence relationships and inclusion relationships. The influence relationships include direct influence and inverse influence.

2. The method according to claim 1, wherein, Generating a target query statement based on the at least one candidate query statement includes: The at least one candidate query statement is input into the large-scale language model, so that the large-scale language model obtains the candidate query statement with the highest matching degree to the first task from the at least one candidate query statement; The target query statement is generated based on the constraints in the first task and the candidate query statement with the highest matching degree.

3. The method according to claim 2, wherein, Based on the constraints in the first task and the candidate query statement with the highest matching degree, the target query statement is generated, including: The limiting conditions in the first task and the candidate query statement with the highest matching degree are input into the large-scale language model, so that the large-scale language model updates the original content in the candidate query statement with the highest matching degree that corresponds to the limiting conditions, and thus obtains the target query statement.

4. The method according to claim 1, wherein, The method further includes: Construct the vector database; Based on the first task, the vector database is queried to obtain at least one candidate query statement that matches the first task, including: The first task is sent to the vector database so that the vector database matches the first task with the description information in the vector database; The sample query statements corresponding to at least one description information that matches the first task in the vector database are determined as at least one candidate query statement that matches the first task.

5. The method according to claim 4, wherein, The method further includes: Obtain structured data from the graph database, the structured data including any node and the relationships between any node and its neighboring nodes; Convert the structured data into a natural language description; Based on the natural language description, priority labels are applied to the association relationships between any node and its neighboring nodes.

6. An intelligent question-answering processing device, the device comprising: The task decomposition module is used to decompose the user's problem into tasks using a large-scale language model to obtain the first task. The query module is used to query the vector database according to the first task to obtain at least one candidate query statement that matches the first task; the vector database includes the correspondence between sample query statements and descriptive information; The generation module is used to generate a target query statement based on the at least one candidate query statement; The first acquisition module is used to acquire the dataset corresponding to the target query statement from the business database; The credibility assessment module is used to determine whether the answer information corresponding to the user question can be generated based on the dataset. The second acquisition module is used to acquire at least one second indicator that is related to the first indicator from the graph database when the answer information corresponding to the user question cannot be generated based on the dataset, based on the first indicator in the first task or the user question. The task decomposition module is also used to decompose the at least one second indicator into a second task. The query module, the generation module, the first acquisition module, and the second acquisition module are further configured to continue executing the second task until the answer information corresponding to the user question is generated. When the construction module constructs the vector database, it is specifically used to: obtain at least one table from the business database; for each table in the at least one table, generate a sample query statement corresponding to the table and obtain the metadata of each column in the table; store the sample query statement corresponding to the table and the metadata of each column in the table into the vector database, wherein the metadata of each column in the table is used as descriptive information; The construction module is further configured to: input the descriptive information from the vector database into the large-scale language model, so that the large-scale language model extracts multiple subject elements from the descriptive information and the relationships between different subject elements, wherein the subject elements include fields and indicators; construct the graph database based on the multiple subject elements from the descriptive information and the relationships between different subject elements, wherein each subject element is a node in the graph database, and the relationships between different subject elements are the relationships between different nodes in the graph database; the relationships include influence relationships and inclusion relationships; the influence relationships include positive proportional influence and negative proportional influence.

7. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-5.

8. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method of any one of claims 1-5.

Images