Question and answer method and system based on table data

Abstract

The embodiment of the invention provides a table data-based question and answer method and system. According to the method, a question and answer system obtains question information which is expressed by a user in a natural language and aims at an original table, and obtains a target table obtained by preprocessing the format of the original table, the preprocessing mode is determined based on the performance of the database programming language and enables the query complexity of the database programming language for the target table to be lower than the query complexity of the database programming language for the original table. Further, the question answering system guides a first large language model to generate a target code expressed in a database programming language based on the question information, executes the target code on a target table to obtain answer information corresponding to the question information, and outputs the question answering information to the user; wherein the target code is encoded to query table content used for answering the question information in the target table.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular, to a question-answering method and system based on tabular data. Background Art

[0002] With the continuous development of artificial intelligence technology, using large language models to answer users' questions has become a key part of intelligent interaction systems. When the user's question is about tabular data, generally, the full amount of tabular data needs to be converted into text, and the above text is used as prompt information and input into the large language model together with the user's question, and the large language model generates an answer.

[0003] However, since tabular data is usually numerical data, the large language model needs to perform specific logical operations and / or numerical operations on the tabular data when answering users' questions. The accuracy of the large language model for such operations is poor, and there may be cases where it cannot correctly answer users' questions.

[0004] The content in the background art section is only the information known to the inventor personally, and does not represent that the above information has entered the public domain before the filing date of this disclosure, nor does it represent that it can be the prior art of this disclosure. Summary of the Invention

[0005] This specification provides a question-answering method and system based on tabular data, which can improve the accuracy of the answer information in the scenario of asking questions about tabular data.

[0006] In a first aspect, this specification provides a question-answering method based on tabular data, including: obtaining the user's question information about the original table, where the question information is expressed in natural language; obtaining a target table obtained by preprocessing the format of the original table, where the preprocessing method is determined based on the performance of the database programming language and makes: the query complexity of the database programming language for the target table is lower than that for the original table; guiding a first large language model to generate target code expressed in the database programming language based on the question information, and executing the target code on the target table to obtain the answer information corresponding to the question information, where the target code is encoded to query the table content for answering the question information in the target table; and outputting the answer information to the user.

[0007] In a second aspect, this specification also provides a question-answering system based on tabular data, including at least one storage medium and at least one processor. The at least one storage medium stores at least one instruction set for performing question answering based on tabular data. The at least one processor is communicatively connected to the at least one storage medium. When the at least one processor runs, it reads the at least one instruction set and executes the method according to any one of the above-mentioned first aspects based on the instructions of the at least one instruction set.

[0008] In a third aspect, this specification also provides a computer-readable non-volatile storage medium. The computer-readable non-volatile storage medium stores at least one instruction set. When the at least one instruction set is executed by at least one processor, it implements the method for question answering based on tabular data according to any one of the above-mentioned first aspects.

[0009] As can be seen from the above technical solutions, for the question-answering method and system based on tabular data provided in this specification, after obtaining the question information of the user for the original table, a target table obtained by preprocessing the original table can be obtained. Based on the question information, the first large language model is guided to generate target code expressed in a database programming language, and the target code is executed on the target table to obtain the answer information corresponding to the question information, and then the answer information is output to the user. Among them, the above preprocessing method is determined based on the performance of the database programming language and makes: the query complexity of the database programming language for the preprocessed target table is lower than that for the original table. The reduction of the query complexity means the reduction of the code complexity. That is to say, compared with the table before preprocessing, the large language model only needs to generate relatively simple code to realize the query of the preprocessed table. This substantially reduces the difficulty of the large language model in generating code and reduces the probability of errors in the process of the large language model generating code. Therefore, by preprocessing the tabular data, the accuracy of the code generated by the large language model can be improved, and then the accuracy of the answer information can be improved.

[0010] Other functions of the question-answering method and system based on tabular data provided in this specification will be partially listed in the following description. The creative aspects of the question-answering method and system based on tabular data provided in this specification can be fully explained by practice or use of the methods, devices and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 Shows a schematic diagram of an application scenario of a question - answering system based on tabular data provided according to an embodiment of this specification;

[0013] Figure 2 Shows a schematic diagram of the hardware structure of a computing device provided according to some embodiments of this specification;

[0014] Figure 3 Shows a schematic flow diagram of a question - answering method based on tabular data provided according to an embodiment of this specification;

[0015] Figure 4 Shows a schematic diagram of an interaction interface provided according to an embodiment of this specification; and

[0016] Figure 5 Shows a schematic flow diagram of a question - answering method based on tabular data provided according to another embodiment of this specification. Detailed implementation manners

[0017] The following description provides specific application scenarios and requirements of this specification, aiming to enable those skilled in the art to manufacture and use the content in this specification. For those skilled in the art, various partial modifications to the disclosed embodiments are obvious, and without departing from the spirit and scope of this specification, the general principles defined here can be applied to other embodiments and applications. Therefore, this specification is not limited to the shown embodiments, but has the broadest scope consistent with the claims.

[0018] The terms used here are only for the purpose of describing specific example embodiments and are not restrictive. For example, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" used here may also include the plural forms. When used in this specification, the terms "comprises", "comprising" and / or "having" mean that the associated integers, steps, operations, elements and / or components exist, but do not exclude the existence of one or more other features, integers, steps, operations, elements, components and / or groups, or the addition of other features, integers, steps, operations, elements, components and / or groups in the system / method.

[0019] Considering the following description, these features of this specification and other features, as well as the operations and functions of the related elements of the structure, and the economy of the combination and manufacture of the components can be significantly improved. Referring to the accompanying drawings, all of these form a part of this specification. However, it should be clearly understood that the drawings are only for the purpose of illustration and description and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0020] The flowcharts used in this specification illustrate the operations implemented by the system according to some embodiments in this specification. It should be clearly understood that the operations in the flowchart may not be implemented in sequence. On the contrary, the operations may be implemented in reverse order or simultaneously. In addition, one or more other operations may be added to the flowchart. One or more operations may be removed from the flowchart.

[0021] In this specification, the expression "X includes at least one of A, B, or C" means that X includes at least A, or X includes at least B, or X includes at least C. That is, X may include only any one of A, B, and C, or may include any combination of A, B, and C as well as other possible contents / elements at the same time. Any combination of A, B, and C may be A, B, C, AB, AC, BC, or ABC.

[0022] In this specification, unless otherwise explicitly stated, the association relationship generated between structures may be a direct association relationship or an indirect association relationship. For example, when describing "A is connected to B", unless it is explicitly stated that A is directly connected to B, it should be understood that A may be directly connected to B or indirectly connected to B; for another example, when describing "A is above B", unless it is explicitly stated that A is directly above B (A and B are adjacent and A is above B), it should be understood that A may be directly above B, or A may be indirectly above B (there are other elements between A and B and A is above B). And so on.

[0023] It should be noted that the user data obtained in this specification has been authorized by the user and does not involve user privacy.

[0024] For the convenience of description, the terms that will appear in the following text of this specification are first explained.

[0025] Large language model: Usually used in the field of artificial intelligence in natural language processing (NLP), especially referring to large machine learning models with a large number of parameters and computing resources. Large language models are named because they have a huge number of parameters and a complex network structure. They have strong feature representation and feature understanding capabilities and can better capture patterns and regularities in data when dealing with complex tasks. They are designed and trained to better understand and generate natural language.

[0026] Database programming language: A database programming language refers to a programming language or language extension used to interact with a database, manage, and operate data. These languages allow developers to perform various database operations, such as querying data, inserting, updating, deleting records, and managing database structures (such as tables, indexes, views, etc.).

[0027] Structured Query Language (SQL): It is a standard programming language used to manage and operate relational databases. Relational databases organize data in the form of tables, and each table consists of rows and columns. SQL provides a complete set of instruction sets that allow users to easily perform operations such as creating, reading, updating, and deleting data in these tables. SQL is one of the commonly used database programming languages.

[0028] The question-answering method based on tabular data provided in this specification can be applied to the question-answering scenario based on tabular data. This method can be executed by a question-answering system. For example, a user inputs a question message to the question-answering system, and this question message is used to inquire about relevant content in the tabular data. The question-answering system can output an answer message to the user based on the question message. Among them, the question message input by the user is information expressed in natural language. In some embodiments, a tabular database can be deployed in the question-answering system, and the tabular database can include one or more tables. The user's question message can be a question about a single table in the tabular database or a question about multiple tables in the tabular database.

[0029] In some embodiments, the above-mentioned question-answering system can serve a specific field. In this case, in different fields, the tabular databases deployed in the question-answering system can be different. For example, in the financial field, the tabular database can include financial tables. In the medical field, the tabular database can include medical tables. In the education field, the tabular database can include education tables, etc. In some embodiments, the question-answering system can also serve multiple fields. In this case, the tabular database can include tables from multiple fields.

[0030] As mentioned above, in the question-answering scenario for tabular data, the current solution usually relies on the powerful text analysis capabilities of large language models. For example, the question-answering system can convert the full amount of tabular data into text, input the user's question message and the converted text into the large language model, and generate an answer message by relying on the text analysis capabilities of the large language. However, due to the limitation of the input length of the large language model, the above method is not applicable to scenarios with a large amount of tabular data. In addition, tabular data is usually numerical data, and the large language model needs to perform specific logical operations and / or numerical operations on the tabular data when answering user questions. The accuracy of the large language model for such operations is poor, and there will be cases where it cannot correctly answer user questions.

[0031] To this end, the technical solution provided in this specification can utilize the code generation ability of the large language model to convert the question information expressed by the user in natural language into target code expressed in the database programming language. Furthermore, the question answering system can obtain the answer information by executing the target code in the tabular data. Since the tabular data no longer needs to be input into the large language model, the problem that the application scenario is limited due to the limited input length of the large language model in the current solution is avoided. Therefore, the technical solution provided in this specification can be applied to a wider range of scenarios. In addition, the large language model has excellent code generation ability and can accurately convert the question information into target code expressed in the database programming language. The database programming language is very suitable for processing tabular data. Moreover, the large language model does not need to perform logical operations and / or numerical operations on the tabular data during the code generation process (these operations are performed during the process of the question answering system executing the target code). This enables the technical solution provided in this specification to generate target code with higher accuracy, and further enables the answer information with higher accuracy to be obtained by executing the target code on the tabular data.

[0032] Furthermore, the technical solution provided in this specification can also perform some preprocessing on the format of the tabular data in advance to make the format of the preprocessed table more suitable for querying in the database programming language. That is, the query complexity of the database programming language for the preprocessed table is lower than that for the table before preprocessing. The reduction of the query complexity means the reduction of the code complexity. That is to say, compared with the table before preprocessing, the large language model only needs to generate relatively simple code to achieve the query of the preprocessed target table. This substantially reduces the difficulty of the large language model in generating code and reduces the probability of errors during the code generation process of the large language model. Therefore, by preprocessing the tabular data, the accuracy of the code generated by the large language model can be further improved, and thus the accuracy of the answer information can be further improved.

[0033] It should be noted that the above description of the application scenario is only part of the multiple usage scenarios provided in this specification. Those skilled in the art should understand that when the question answering method and system based on tabular data provided in this specification are applied to other usage scenarios, their implementation manners and technical effects are similar.

[0034] Figure 1 Fig. 100 shows a schematic diagram of an application scenario 100 provided according to an embodiment of this specification. As Figure 1 shown, the application scenario 100 may include a user 110 and a question answering system 130 based on tabular data (hereinafter referred to as system 130).

[0035] In this application scenario 100, a tabular database can be deployed in system 130. In some embodiments, the tabular database can include at least one original table and at least one target table obtained by preprocessing at least one original table. The preprocessing method can be referred to the relevant description later and will not be elaborated here. The original tables and the target tables are in one-to-one correspondence. The content of an original table and its corresponding target table is the same, but their formats are different. For example, the original table adopts the table format that people are used to reading in the actual scenario. While the target table adopts the table format suitable for querying by database programming languages. In some embodiments, the tabular database can also only include the preprocessed target tables. The original tables can be stored in other databases.

[0036] Continuing to refer to Figure 1 , a large language model is also deployed in system 130. The large language model has the ability to generate code expressed in a database programming language. User 110 can input a question message to system 130, which is expressed in natural language and used to query relevant content in the original table. After obtaining the question message, system 130 can obtain the target table corresponding to the original table, guide the large language model to generate target code expressed in a database programming language based on the question message, and execute the target code on the target table to obtain the answer message corresponding to the question message. Furthermore, system 130 can output the answer message to user 110.

[0037] In some embodiments, system 130 can be an electronic device with certain computing capabilities. The question-answering method based on tabular data can be executed on system 130, specifically by the processor in system 130. At this time, system 130 can store the data or instructions for executing the question-answering method based on tabular data described in this specification and can execute or be used to execute the data or instructions. System 130 can include a hardware device with the question-answering function based on tabular data and the program required to drive the hardware device to work.

[0038] System 130 can be a single computing device or a cluster system composed of multiple computing devices, which is not limited in this specification.

[0039] It should be noted that all user data obtained in this specification has been authorized by the users and does not involve user privacy.

[0040] Figure 2 Shows a schematic diagram of the hardware structure of a computing device 200 provided according to some embodiments of this specification. The computing device 200 can be used as Figure 1 the system 130 in. In some embodiments, when system 130 adopts a device cluster, the computing device 200 can be used as any device in system 130.

[0041] As Figure 2 shown, the computing device 200 includes at least one storage medium 230 and at least one processor 220. In some embodiments, the computing device 200 may further include an internal communication bus 210. In some embodiments, the computing device 200 may further include a communication port 250. In some embodiments, the computing device 200 may further include I / O components 260.

[0042] The internal communication bus 210 can connect different system components, including the storage medium 230 and the processor 220. The I / O components 260 support input / output between the computing device 200 and other components.

[0043] The communication port 250 is used for data communication between the computing device 200 and the outside world. For example, the computing device 200 can be connected to a network through the communication port 250.

[0044] The storage medium 230 may include a data storage device. The data storage device can be a non-transitory storage medium or a transitory storage medium. For example, the data storage device may include one or more of a magnetic disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 236. The storage medium 230 further includes at least one instruction set stored in the data storage device. The instruction set is computer program code, and the computer program code may include programs, routines, objects, components, data structures, procedures, modules, etc. for executing the question-answering method based on tabular data provided in this specification.

[0045] The at least one processor 220 is communicatively connected to the at least one storage medium 230 through the internal communication bus 210. The at least one processor 220 is configured to execute the above at least one instruction set. When the system 130 runs, the at least one processor 220 reads the at least one instruction set and executes the question-answering method based on tabular data provided in this specification according to the instructions of the at least one instruction set.

[0046] The processor 220 may execute all steps included in the question-answering method based on tabular data. The processor 220 may be in the form of one or more processors. The processor 220 may issue execution instructions. The processor 220 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, etc., or any combination thereof.

[0047] For illustrative purposes only, only one processor 220 is shown in the computing device 200 in the attached drawings in this specification. However, it should be noted that the computing device 200 in this specification may also include multiple processors, and therefore, the operations and / or method steps disclosed in this specification may be performed by one processor as in this specification, or may be performed jointly by multiple processors. For example, if the processor 220 of the computing device 200 in this specification performs step A and step B, it should be understood that step A and step B may also be performed jointly or separately by two different processors 220 (e.g., the first processor performs step A, the second processor performs step B, or the first and second processors perform steps A and B together).

[0048] Figure 3 FIG. 1 is a flow chart of a question-answering method based on table data provided according to an embodiment of the present specification; the question-answering method based on table data P300 may be executed by the system 130. Figure 3 As shown, the method P300 provided in this specification may include S310-S370.

[0049] S310: Obtaining the user's question information regarding the original form, where the question information is expressed in natural language.

[0050] The question information is used to inquire about the relevant content in the original table. For example, in the financial mutual evaluation scenario, the question information entered by the user can be: "According to the latest mutual evaluation results, which jurisdictions scored 85 points in evaluation item 13?", or "In the latest mutual evaluation results, what is the score of China in each evaluation item?". In the above examples, "the latest mutual evaluation results" represents the original table.

[0051] The above question information can be in text form or voice form. When the question information is in voice form, after obtaining the question information, system 130 can first convert it into text form and then perform subsequent steps.

[0052] Figure 4 FIG. shows a schematic diagram of an interaction interface provided according to an embodiment of the present specification, as Figure 4 shown, an input field for question information can be displayed in the interaction interface 430. The user can enter question information in the input field for question information. After the user finishes entering the question information, the user can trigger the input of the user's question into the system 130 by clicking the send control. The way for the user to enter question information can be to directly enter text information; or, alternatively, it can be by voice input, obtaining the voice information input by the user and converting the voice information into text information and displaying it in the input field. It should be understood that the specific way for the user to enter question information can be flexibly adjusted according to the actual scenario and is not limited to that given in the above embodiment.

[0053] In some embodiments, one or more reference question information can also be displayed in the interaction interface 430 to guide the user to enter the question information expected by the user through the reference question information. The reference question information can be displayed in a classified form, as Figure 4 shown, taking the target table as a financial risk assessment table as an example for illustration (the target table includes the scores of multiple objects under multiple risk assessment items), for example, the types of reference question information can include at least one of a full-text summary type (full-text summary questions), a single-item summary type (single-item summary questions), a comparison type (comparison questions), or a semantic type (semantic questions). For each type, one or more reference question information corresponding to that type can be displayed in the interaction interface 430.

[0054] The user can select question information from at least one reference question information and directly input the selected question information into the system 130 to guide the large language model to generate the target code based on the question information selected by the user. Or, after the user selects question information from at least one question information, the selected question information will be displayed in the input field for question information. The user can modify the reference question information in the input field. After the modification is completed, the user then clicks the send control to trigger the input of the modified question information into the system 130.

[0055] S330: Obtain a target table obtained by preprocessing the format of the original table, where the preprocessing method is determined based on the performance of the database programming language and makes: the query complexity of the database programming language for the target table is lower than that for the original table.

[0056] Combined with Figure 1In the scenario shown, system 130 can obtain a target table obtained by preprocessing the format of the original table from a tabular database. It should be noted that the query information obtained by system 130 in S310 can be a query for one original table or a query for multiple original tables. When the query information is a query for multiple original tables, the multiple target tables corresponding to the multiple original tables obtained by system 130 in S330.

[0057] In some embodiments, the process of preprocessing the format of the original table can be pre-executed before S310. For example, before system 130 goes online, all original tables are preprocessed separately to obtain target tables, and all the obtained target tables are maintained in the tabular database. In this way, after system 130 obtains the query information of the user for the original table, it can directly obtain the target table corresponding to the original table from the tabular database in S330. In some embodiments, the process of preprocessing the format of the original table can also be performed in real time. For example, after system 130 obtains the query information of the user for the original table, it preprocesses the format of the original table in S330 to obtain a target table.

[0058] In this specification, the preprocessing method is determined based on the performance of the database programming language, and its purpose is to reduce the query complexity of the database programming language for the table. That is, the preprocessing method makes: the query complexity of the database programming language for the preprocessed table (i.e., the target table) is lower than that for the table before preprocessing (i.e., the original table). The reduction of the query complexity includes at least one of the following: the number of steps for querying using the database programming language is reduced, or the number of rows or columns to be processed in each step is reduced. The reduction of the query complexity means the reduction of the complexity of the query code. For example, the query code of the database programming language for the original table includes 20 statements, while the query code for the target table includes 5 statements. Another example is that the query code of the database programming language for the original table includes 20 row and column filtering operations, while the query code for the target table only includes 5 row and column filtering operations.

[0059] Table data usually includes multiple rows and multiple columns. In some embodiments, preprocessing at least includes: row-column conversion processing. For example, the row-column conversion processing can be: splitting one row into multiple rows, reducing the number of columns and increasing the number of rows. That is to say, the number of rows of the target table after preprocessing is greater than the number of rows of the original table, and the number of columns of the target table after preprocessing is less than the number of columns of the original table. Another example, the row-column conversion processing can be: merging multiple rows into one row, reducing the number of rows and increasing the number of columns. That is to say, the number of columns of the target table after preprocessing is greater than the number of columns of the original table, and the number of rows of the target table after preprocessing is less than the number of rows of the original table. It should be understood that the preprocessing is only for processing the table format of the original table, and the table content included in the target table is the same as that of the original table, only the table formats are different.

[0060] Since the SQL language has powerful expressive power, efficient execution ability, precise query intention description, as well as good compatibility and maintainability, it can be compatible with multiple database systems, can efficiently and accurately process complex query requirements, and at the same time ensure the query efficiency and the accuracy of the results. Therefore, in some embodiments, the database programming language can adopt the SQL language. When the database programming language is the SQL language, the way of row-column conversion processing can be: reducing the number of columns and increasing the number of rows, corresponding multiple rows of data in the target table to one row of data in the original table, so that the number of columns of the target table is less than the number of columns of the original table.

[0061] When the SQL language processes a table, it supports column filtering operations and row filtering operations. The following gives examples of column filtering operations and row filtering operations.

[0062] The purpose of column filtering is to select specific columns from the table. The column filtering operation can be implemented by the following statement:

[0063] SELECT column1,column2,……

[0064] FROM table_name;

[0065] column1, column2 are the column names to be retrieved, which can be one column or multiple columns. If all columns are to be retrieved, the wildcard * can be used to represent. table_name represents the table name.

[0066] The purpose of row filtering is to select rows that meet specific conditions from the table. The row filtering operation can be implemented by the following statement:

[0067] SELECT column1,column2,……

[0068] FROM table_name

[0069] WHERE condition;

[0070] Among them, the meanings of column1, column2, and table_name are similar to those above. condition is a conditional expression used to specify the filtering conditions for the query. Only the rows that meet this condition will be returned. For example, WHERE age > 25 means only the records with an age greater than 25 will be returned.

[0071] Although the basic syntax of row filtering and column filtering may not seem complex, in practical applications, column filtering often involves complex operations such as calculating, aggregating, and subquerying column data. These operations increase the code complexity of column filtering. If column filtering also involves multi-table joins, nested queries, etc., the code will become even more complex.

[0072] It can be seen from this that the code generated by the SQL language for row filtering of a table is relatively simple, but the code generated for column filtering of a table is relatively complex. Therefore, the target table generated by the preprocessing method of "reducing the number of columns and increasing the number of rows" has an increased number of rows and a decreased number of columns compared to the original table. Compared with the original table, the complexity of querying the target table using the SQL language is reduced. The reduced complexity of the SQL language means that the large language model only needs to generate relatively simple code to achieve querying of the preprocessed target table. This substantially reduces the difficulty of the large language model in generating code and reduces the probability of errors in the process of the large language model generating code. Therefore, by the above method of reducing the number of columns and increasing the number of rows in the original table, the accuracy of the code generated by the large language model can be further improved, thereby further improving the accuracy of the answer information.

[0073] Still taking the database programming language as SQL as an example, Table 1 shown below is the original table, which is a risk assessment summary table in a financial scenario. The meanings of each column name from left to right in this original table are: jurisdiction, report_type, report_date, assessment_body, score of assessment item 1 (item1 score), score of assessment item 2 (item2 score), ……, score of assessment item n (itemn score).

[0074] Table 1:

[0075]

[0076] The target table shown in Table 2 below. The target table combines the scores of Evaluation Item 1, Evaluation Item 2, …, Evaluation Item n in Table 1 into a column named evaluation_term, and adds a new column named rating to represent the score of each evaluation item.

[0077] Table 2:

[0078]

[0079] It can be seen that after reducing the number of columns and increasing the number of rows in the table corresponding to Table 1, the table corresponding to Table 2 will be obtained. In the case of a large number of evaluation items, the number of columns in Table 2 will be greatly reduced compared to Table 1. For example, if there are 50 evaluation items in Table 1, that is, the number of columns related to evaluation items in Table 1 is 50 columns. After reducing the number of columns and increasing the number of rows as described above, the content related to the 50 columns of evaluation items will be integrated into 2 columns to obtain Table 2. Obviously, the number of columns in Table 2 is greatly reduced compared to Table 1 (reduced by 50 - 2 = 48 columns), and the difficulty of generating SQL statements by the large language model based on Table 2 will be greatly reduced.

[0080] In some embodiments, the preprocessing further includes at least one of removing useless information or standardizing column names. Among them, the column name standardization process is used to convert the column names in the table into a form that conforms to the coding specifications of the database programming language.

[0081] Among them, since in addition to the table data, the table may also contain some descriptive data (large paragraphs of text narrative or irregular character combinations, etc.). During the preprocessing process, it is necessary to remove this type of descriptive data that does not conform to the table format in the table and only retain the data in the table format, so that the processed table can be automatically processed by the database programming language, thereby improving the efficiency and accuracy of subsequent table data processing.

[0082] Column name standardization processing may include: symbol standardization processing and character standardization processing. Among them, symbol standardization processing may be to modify all spaces in the table to underscore symbols. Character standardization processing may adjust all alphabetic characters in the table to lowercase English letters and delete repetitive characters (for example, for multiple characters connected in the form of "or", only one of the multiple characters may be retained). For example, the column names included in the table are as follows: Jurisdiction, Report Type, Report Date, Assessment body / bodies, etc. After column name standardization processing, each column name is respectively modified to: jurisdiction, report_type, report_date, assessment_body. The column names in the table after column name standardization processing only contain lowercase English letters and underscore characters, reducing the complexity of the column names in the table, unifying the format of the column names, making the column names in the table more regular and unified, meeting the processing requirements of the database programming language, and thus reducing the probability of errors when the large language model generates code.

[0083] S350: Based on the question information, guide the first large language model to generate target code expressed in the database programming language, and execute the target code on the target table to obtain the answer information corresponding to the question information, where the target code is encoded to query the table content used to answer the question information in the target table.

[0084] In some possible embodiments, the system 130 generates a guidance instruction at least based on the question information, and inputs the guidance instruction into the first large language model to guide the first large language model to generate target code. Then, the system 130 executes the target code on the target table to obtain an execution result.

[0085] Figure 5 Shows a schematic flowchart of a question-answering method based on table data provided by another embodiment of this specification, as Figure 5 As shown, in the case where the execution result of the target code is execution failure, the system 130 may update the guidance instruction and input the updated guidance instruction into the first large language model to guide the first large language model to generate updated target code. Subsequently, the system 130 executes the updated target code on the target table to obtain an execution result. If the execution result is still execution failure, the system 130 repeats the above steps of updating the guidance instruction until the execution result of the target code is execution success. The system 130 obtains the query result returned by the target code with successful execution and obtains the answer information corresponding to the question information based on the query result.

[0086] Among them, in the case where the execution result is a failure, the system 130 can obtain the failure error message corresponding to the current target code. Then, based on the current target code and the failure error message, the system 130 updates the guiding instruction to obtain the updated guiding instruction. The way to update the guiding instruction can be: the system 130 splices the currently failed target code and the failure error message corresponding to the target code, and supplements the spliced information into the guiding instruction to form the updated guiding instruction.

[0087] After the above method fails to execute the current target code, the system 130 reduces the probability of being unable to obtain the answer information due to the generation error of the target code by continuously updating the guiding instruction and re-trying to generate a new target code based on the guiding instruction, improving the success rate of the system 130 when processing various question information, and thus improving the stability of the system 130. In addition, by supplementing the currently failed target code and the failure error message into the guiding instruction and inputting the updated guiding instruction into the first large language model, the system 130 can provide richer and more specific feedback information for the first large language model based on the updated guiding instruction. Based on the above feedback information, the first large language model can understand why the target code generated in the previous round failed to execute, so as to make targeted adjustments when generating the target code in the subsequent process to generate a target code that more conforms to the question intention, thereby improving the accuracy of target code generation.

[0088] In some possible embodiments, to ensure the instant feedback of the question information, when the number of execution failures is greater than or equal to the preset number of times, the system 130 can output a prompt message to the user, and the prompt message is used to indicate that the system 130 cannot answer the question information. By setting the preset number of execution failures, the number of times the system attempts to answer the question information is effectively limited, avoiding the continuous consumption of system resources such as computing resources and memory caused by the system 130 continuously attempting to answer the question information, avoiding the ineffective waste of system resources, helping to maintain the stable operation of the system 130, and ensuring the normal use of other users. At the same time, when the number of execution failures is greater than or equal to the preset number of times, outputting a prompt message to the user in a timely manner can also avoid the problem of the user waiting continuously caused by the long-term execution failure, improving the user experience.

[0089] Such as Figure 5As shown, to improve the accuracy of the target code, system 130 can obtain at least one reference example based on the question information. Each reference example includes a reference question and a reference code. System 130 generates a guiding instruction based on the at least one reference example and the question information, and inputs the guiding instruction into the first large language model to guide the first large language model to generate the target code. Among them, the guiding instruction is used to guide the first large language model to use the at least one reference example as a reference when generating code for the question information. During the process of the first large language model generating the target code, due to the guidance of relevant reference examples, more relevant content will be included in the guiding instruction, and the target code generated by the first large language model based on such a guiding instruction will be more in line with the actual requirements, and the accuracy will also be improved. By adding reference examples to the guiding instruction, system 130 makes the content included in the generated guiding instruction more, and the first large language model can quickly generate the target code that meets the requirements based on the patterns and ideas of the reference examples in the guiding instruction, which not only improves the accuracy of the target code generated by the first large language model, but also improves the efficiency of the first large language model in generating the target code.

[0090] Among them, the way to obtain at least one reference example can be: system 130 obtains at least one reference example from a preset knowledge base based on the question information. The preset knowledge base includes multiple sample examples, each sample example includes a sample question and a sample code. The sample question is a question for a table, and the sample code is configured to query the table content required to answer the sample question from the corresponding table. Among them, the similarity between the sample question in each reference example and the question information meets the preset conditions.

[0091] It should be understood that the reference examples are obtained by system 130 from the preset knowledge base based on the question information, and different question information may correspond to different reference examples. The above reference examples can also be understood as the learning samples of the large language model. System 130 dynamically determines at least one reference example based on the question information, and generates a guiding instruction based on the at least one reference example and the question information. The generation process of this guiding instruction can also be called a dynamic few-shot prompt construction process. By adding at least one reference example to the guiding instruction, system 130 can enrich the data content in the guiding instruction and guide the first large language model to generate the target code in the correct direction. That is to say, for the question information of each user, system 130 can obtain the corresponding reference example from the preset knowledge base, so that the guiding instruction can be dynamically adjusted according to different question information, thereby improving the accuracy of the code generated by system 130.

[0092] When determining at least one reference example, system 130 may determine a first representation corresponding to the question information. The first representation refers to a form of representation or feature vector corresponding to the question information, and it is a way of expressing the key features, semantics, etc. of the question information. In addition, for each example in the preset knowledge base, system 130 may determine a second representation corresponding to the sample question in the example. The second representation is a form of representation or feature vector, etc. corresponding to the sample question in each example in the preset knowledge base. Similar to the first representation, the second representation is obtained after processing the sample question and is a way of expressing the core features, semantics, etc. of the sample question.

[0093] System 130 determines the similarity between the question information and the sample questions in the examples based on the first representation and the second representation, and sorts the multiple examples in the preset knowledge base in descending order of similarity. At least one example with a higher ranking is determined as a reference example. Since the similarity between the question information and the sample questions in the reference examples meets the preset requirements in the above-mentioned way of determining reference examples, the guiding instructions generated based on the above reference examples and the question information can be more closely aligned with the user's question intention. When the large language model generates target code based on such guiding information, the accuracy of the generated target code will be significantly improved.

[0094] In some possible embodiments, after system 130 sorts multiple sample examples from high to low based on the first representation and the second representation, it may also randomly determine a preset number of examples as reference examples among at least one example with a higher ranking according to the preset number of reference examples. For example, after system 130 sorts multiple examples in the preset knowledge base in descending order of similarity, the similarities of the top 5 examples in the similarity ranking are the same as that of the question information, and the preset number of reference examples is 3. Then system 130 may randomly select 3 examples from the top 5 examples as reference examples. Alternatively, the preset condition may also be a preset similarity threshold. System 130 obtains multiple examples with similarities greater than the similarity threshold as candidate examples, and then randomly obtains a preset number of examples as reference examples among the candidate examples based on the preset number of reference examples. It should be understood that the above embodiments are only for illustrative purposes, and the specific way of determining reference examples can be flexibly adjusted according to user needs and is not limited to those given in the above embodiments.

[0095] In some other possible embodiments, to improve the pertinence of the target code, the system 130 may further generate task description information based on the table structure information of the target table. The table structure information at least includes the header information in the target table. Then, the system 130 generates guiding instructions based on at least one reference example, the question information, and the task description information. The header information can clearly define the meaning and scope of the data in each column of the target table. Adding the header when generating the guiding instructions can make the guiding instructions more accurately match the operation requirements of the question information for the table data, ensure that the target code processes specific data columns as expected, and avoid the situation of incorrect or deviated operation objects. In addition, since different target tables may involve different table structures, the method of generating guiding instructions by the system 120 based on the table structure information can make the target code generated by the first large language model more suitable for querying the current target table, thereby improving the accuracy of the target code.

[0096] In some embodiments, the system 130 may generate task description information based on the table structure information of the target table and the task background information. The task background information is used to describe the information related to the application scenario corresponding to the target table. For example, in a medical scenario, the task background information may include content related to clinical diagnosis and treatment analysis, medical research analysis, or medical management and quality control. Adding the task background information to the guiding instructions can more clearly provide more relevant information to the first large language model. When the first large language model clarifies the current task background, the generated target code will be more accurate.

[0097] As Figure 5 shown, since the query results returned by the target code are usually presented in a specific data format of the target table, the query results usually retain the original data format (for example, presented in a two-dimensional table form, including rows and columns, and the data type strictly follows the definition of the target table). To ensure the readability and understandability of the answer information, after obtaining the query results returned by the target code, the second large language model may be used to generate answer information based on the question information and the query results. The answer information is information expressed in natural language. The first large language model and the second large language model may be the same large language model or different large language models. For example, the first large language model may be selected as a large language model with better performance in generating database programming languages, and the second large language model may be selected as a large language model with better effects in generating natural language. The specific selection method of the first large language model and the second large language model can be flexibly adjusted according to user needs and is not limited to those given in the above embodiments.

[0098] During the process of generating response information by the second largest language model, the query results will be analyzed, refined, and / or summarized based on the query information to generate response information that meets the user's expectations. For example, if the user's query information is to inquire about the workload of each department in the company, the query results may include the specific data of the workload of each employee in the company. Obviously, if the specific data of the workload of each employee is fed back to the user, the readability for the user is poor. By using the method provided in this specification, after obtaining the query results, the query results will also be processed based on the second largest language model and response information will be output. The processing method can be summarization and sorting. The response information output after being processed by the second largest language model may only include key information such as the summary of the workload of each department and the workload ranking. The above way of converting the query results into natural language enables the response information to be presented to the user in a way that conforms to the user's communication habits, thereby improving the fluency and naturalness in the interaction process between the user and system 130.

[0099] S370: Output response information to the user.

[0100] After the user obtains the response information, the user can view the response information. If the response information does not meet the user's expectations, the user can obtain new response information by asking questions again.

[0101] As Figure 5 shown, while outputting response information to the user, feedback rating options for the response information can also be output to the user. The user can give feedback ratings to the response information corresponding to each query information based on the feedback rating options. When the feedback rating is greater than or equal to the preset threshold, it means that the user approves of the current response information, indicating that the target code generated based on the query information meets the user's needs. Then system 130 can generate new sample examples based on the query information and the target code, and add the new sample examples to the preset knowledge base. The above way of adding the query information and the target code with feedback ratings greater than the preset threshold to the preset knowledge base adds high-quality sample examples to the preset knowledge base, enabling the preset knowledge base to gradually enrich and improve the types and quantities of samples in the preset knowledge base as the user uses it, which helps to improve the overall quality of the preset knowledge base, thus better adapting to various complex and changeable user needs and improving the adaptability and stability of system 130 in different scenarios.

[0102] For the question information and target code with a feedback score less than the preset threshold, the system 130 can feedback them to the terminal of the operation and maintenance personnel. The operation and maintenance personnel correct the target code to correct the problems existing in the target code (such as logical error problems or algorithm error problems, etc.), and supplement the question information and the corrected target code into the preset knowledge base. In the above method, correcting the target code whose feedback score does not meet the preset requirements is equivalent to providing high-quality content for the preset knowledge base, ensuring the accuracy and reliability in the preset knowledge base, and effectively avoiding the accumulation of errors or low-quality information. On the premise of ensuring the quality of the examples in the preset knowledge base (each example includes a question information and the corresponding target code), with the continuous use of users, a large number of new sample examples that meet the requirements are continuously added. The knowledge coverage of the preset knowledge base is continuously expanded, covering more different types of question information and the corresponding target code, and the knowledge system of the preset knowledge base will be more complete, which can better handle various complex and changeable user needs and enhance the adaptability of the preset knowledge base to different scenarios.

[0103] In summary, in the question and answer method P300 and system 130 based on tabular data provided in this specification, after obtaining the question information of the user for the original table, the target table obtained by preprocessing the original table can be obtained. Based on the question information, the first large language model is guided to generate the target code expressed in the database programming language, and the target code is executed on the target table to obtain the answer information corresponding to the question information, and then the answer information is output to the user. Among them, the above preprocessing method is determined based on the performance of the database programming language and makes: the query complexity of the database programming language for the preprocessed target table is lower than that for the original table. The reduction of the query complexity means the reduction of the code complexity. That is to say, compared with the table before preprocessing, the large language model only needs to generate relatively simple code to realize the query of the preprocessed table. This essentially reduces the difficulty of the large language model in generating code and reduces the probability of the large language model making mistakes in the process of generating code. Therefore, by preprocessing the tabular data, the accuracy of the code generated by the large language model can be improved, and then the accuracy of the answer information can be improved.

[0104] On the other hand, this specification provides a computer-readable non-transitory storage medium storing at least one instruction set for performing question and answer based on tabular data. When the at least one instruction set is executed by a processor, the at least one instruction set instructs the processor to implement the steps of the question and answer method P300 based on tabular data in this specification. In some possible implementation manners, various aspects of this specification can also be implemented in the form of a program product, which includes program code. When the program product runs on the system 130, the program code is used to cause the system 130 to execute the steps of the method P300 described in this specification. The program product for implementing the above method can be a portable compact disc read-only memory (CD-ROM) including program code and can run on the system 130. However, the program product of this specification is not limited thereto. In this specification, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system. The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. The program code for performing the operations of this specification can be written in any combination of one or more programming languages, including object-oriented programming languages - such as Java, C++, etc., and also including conventional procedural programming languages - such as the "C" language or similar programming languages.

[0105] The above description has been made of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require a particular order or a sequential order to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0106] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented by way of example only and is not necessarily limiting. Although not explicitly stated herein, those skilled in the art will understand that this specification is intended to encompass various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be proposed by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0107] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" mean that the particular features, structures, or characteristics described in connection with that embodiment may be included in at least one embodiment of this specification. Thus, it should be emphasized and understood that two or more references to "an embodiment" or "one embodiment" or "alternative embodiments" in various parts of this specification do not necessarily all refer to the same embodiment. Additionally, the particular features, structures, or characteristics may be appropriately combined in one or more embodiments of this specification.

[0108] It should be understood that in the foregoing description of the embodiments of this specification, for the purpose of helping to understand a feature, and for the purpose of simplifying this specification, this specification combines various features in a single embodiment, drawing, or its description. However, this does not mean that the combination of these features is necessary, and it is entirely possible for those skilled in the art, when reading this specification, to mark out some of the devices as separate embodiments for understanding. That is to say, the embodiments in this specification can also be understood as the integration of multiple sub - embodiments. And the content of each sub - embodiment is also valid when it has fewer features than all the features of a single foregoing disclosed embodiment.

[0109] Each patent, patent application, published patent application, and other materials cited in this disclosure, such as articles, books, specifications, publications, documents, references, etc. (excluding any historical prosecution files associated therewith), are hereby incorporated by reference for all purposes relevant to this disclosure, e.g., in the specification and claims of this disclosure. However, if there is any inconsistency or conflict between the descriptions, definitions, and / or terms of such materials and those used in this disclosure, the descriptions, definitions, and / or terms used in this disclosure shall prevail.

[0110] Finally, it should be understood that the embodiments of the applications disclosed herein are illustrative of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art may implement the applications in this specification by adopting alternative configurations based on the embodiments in this specification. Therefore, the embodiments of this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A question-answering method based on tabular data, comprising: Obtaining question information from a user regarding the original form, where the question information is expressed in natural language; Obtaining a target table obtained by preprocessing the format of the original table, wherein the preprocessing method is determined based on the performance of a database programming language and makes: the query complexity of the database programming language for the target table lower than the query complexity for the original table; Based on the question information, guiding the first language model to generate a target code expressed in the database programming language, and executing the target code on the target table to obtain answer information corresponding to the question information, wherein the target code is encoded to query the table content in the target table for answering the question information; and The answer information is output to the user.

2. The method of claim 1, wherein: The preprocessing method at least includes: row-column conversion processing.

3. The method of claim 2, wherein: The database programming language is structured query language SQL; The row-column conversion process is to reduce the number of columns and increase the number of rows. The multiple rows of data in the target table correspond to one row of data in the original table, and the number of columns of the target table is less than the number of columns of the original table.

4. The method of claim 2, wherein: The preprocessing method also includes at least one of removing useless information or standardizing column names. The column name standardization process is used to convert the column names in the table into a form that complies with the coding specification of the database programming language.

5. The method of claim 1, wherein: The step of guiding the first language model to generate target code expressed in the database programming language based on the question information includes: Obtain at least one reference example based on the question information, each reference example including a reference question and a reference code; generating a guidance instruction based on the at least one reference example and the question information, the guidance instruction being used to guide the first large language model to use the at least one reference example as a reference when generating code for the question information; and The guiding instruction is input into the first large language model to guide the first large language model to generate the target code.

6. The method of claim 5, wherein: The method further comprises: Generate task description information based on table structure information of the target table, wherein the table structure information at least includes table header information in the target table; The generating of the guidance instruction based on the at least one reference example and the question information comprises: The guidance instruction is generated based on the at least one reference example, the question information, and the task description information.

7. The method of claim 5, wherein: The obtaining at least one reference example based on the question information includes: Based on the question information, the at least one reference example is obtained from a preset knowledge base, wherein the preset knowledge base includes a plurality of sample examples, each of which includes a sample question and a sample code. Wherein, the similarity between the sample question in each reference example and the question information meets a preset condition.

8. The method of claim 7, wherein: The obtaining the at least one reference example from a preset knowledge base based on the question information includes: Determining a first representation corresponding to the question information; For each example in the preset knowledge base, determine a second representation corresponding to a sample question in the example, and determine a similarity between the question information and the sample question in the example based on the first representation and the second representation; Sorting the multiple examples in the preset knowledge base in descending order of the similarity; and At least one example with a higher ranking is determined as the reference example.

9. The method of claim 1, wherein: The step of guiding the first language model to generate a target code based on the question information, and executing the target code on the target table to obtain answer information corresponding to the question information includes: generating a guiding instruction based at least on the question information, inputting the guiding instruction into the first large language model to guide the first large language model to generate a target code, and executing the target code on the target table to obtain an execution result; When the execution result is an execution failure, the following steps are repeated until the execution result is a success, the query result returned by the target code is obtained, and the answer information corresponding to the question information is obtained based on the query result: The guiding instruction is updated, and the updated guiding instruction is input into the first large language model to guide the first large language model to generate an updated target code, and the updated target code is executed on the target table to obtain an execution result.

10. The method of claim 9, wherein: The method further comprises: when the execution result is a failure, obtaining failure error information corresponding to the current target code; The updating of the boot instruction comprises: Based on the current target code and the failure error information, the boot instruction is updated to obtain an updated boot instruction.

11. The method of claim 9, wherein: The method further comprises: When the number of execution failures is greater than or equal to a preset number, a prompt message is output to the user, wherein the prompt message indicates that the question information cannot be answered.

12. The method of claim 9, wherein: The obtaining answer information corresponding to the question information based on the query result includes: The answer information is generated based on the question information and the query result using a second language model.

13. The method of claim 1, wherein: The method further comprises: Obtaining a feedback score from the user on the answer information; and In a case where the feedback score is greater than a preset threshold, a new sample example is generated based on the question information and the target code, and the new sample example is added to a preset knowledge base.

14. A question-answering system based on tabular data, comprising: at least one storage medium storing at least one instruction set for performing question-answering based on table data; as well as At least one processor is communicatively connected to the at least one storage medium, wherein the at least one processor reads the at least one instruction set when running, and executes the method described in any one of claims 1-13 according to the instructions of the at least one instruction set.

Images