A method, device and equipment for obtaining question and answer pairs

Abstract

The present application provides a method, device and equipment for obtaining a question-answer pair, the method comprising: obtaining the row title, column title and cell content corresponding to the candidate cell; generating a first question and a second question based on the row title and the column title, wherein the row title in the first question is located in front of the column title, and the column title in the second question is located in front of the row title; determining a reference question based on the semantic coherence of the first question and the semantic coherence of the second question, if the semantic coherence of the first question is greater than that of the second question, the reference question is the first question; if the semantic coherence of the first question is less than that of the second question, the reference question is the second question; generating an extended question-answer pair based on the reference question and the cell content, wherein the question of the extended question-answer pair is the reference question, and the answer to the extended question-answer pair is the cell content; generating a target question-answer pair based on the extended question-answer pair. Through the scheme of the present application, the diversity of question-answer pairs is ensured, and the generation quality of question-answer pairs is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device and equipment for obtaining question and answer pairs. Background Art

[0002] With the rapid development of network technology, there is more and more network data. In order to retrieve the data required by users from a large amount of network data, users are required to provide original input questions. The server retrieves the data required by users from a large amount of network data based on the original input questions and returns the data to the user.

[0003] However, due to the large amount of network data, the server needs to consume a lot of computing resources to retrieve the data required by the user from the large amount of network data, and the server needs to spend a lot of time, resulting in a large amount of server resource consumption, and the user can only obtain the required data for a long time. Summary of the invention

[0004] The present application provides a method for obtaining a question-answer pair, the method comprising:

[0005] Acquire original input data; if the original input data includes original table input data, for each candidate cell in the original table input data, acquire the row title, column title and cell content corresponding to the candidate cell from the original table input data;

[0006] Generate a first question and a second question corresponding to the candidate cell based on the row header and the column header; wherein, in the first question, the row header is located before the column header, and in the second question, the column header is located before the row header;

[0007] Determine a reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question; wherein, if the semantic coherence of the first question is greater than the semantic coherence of the second question, the reference question is the first question; if the semantic coherence of the first question is less than the semantic coherence of the second question, the reference question is the second question;

[0008] Generate an extended question-answer pair based on the reference question and the cell content; wherein the question of the extended question-answer pair is the reference question, and the answer to the extended question-answer pair is the cell content;

[0009] A target question-answer pair corresponding to the original input data is generated based on the expanded question-answer pair.

[0010] The present application provides a question-answer pair acquisition device, the device comprising:

[0011] an acquisition module, configured to acquire original input data; if the original input data includes original table input data, for each candidate cell in the original table input data, acquire the row title, column title and cell content corresponding to the candidate cell from the original table input data;

[0012] A generating module, configured to generate a first question and a second question corresponding to the candidate cell based on the row header and the column header; wherein, in the first question, the row header is located before the column header, and in the second question, the column header is located before the row header;

[0013] A determination module, configured to determine a reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question; wherein, if the semantic coherence of the first question is greater than the semantic coherence of the second question, the reference question is the first question; and if the semantic coherence of the first question is less than the semantic coherence of the second question, the reference question is the second question;

[0014] A processing module is used to generate an extended question-answer pair based on the reference question and the cell content; wherein the question of the extended question-answer pair is the reference question, and the answer to the extended question-answer pair is the cell content; and a target question-answer pair corresponding to the original input data is generated based on the extended question-answer pair.

[0015] The present application provides an electronic device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned question-answer pair acquisition method.

[0016] The present application provides a computer program product, which may include a computer program. When the computer program is executed by a processor, the above-mentioned question-answer pair acquisition method is implemented.

[0017] The present application provides a machine-readable storage medium, which stores machine-executable instructions that can be executed by a processor; wherein the processor is used to execute the machine-executable instructions, and when the machine-executable instructions are executed, the above-mentioned question-answer pair acquisition method is implemented.

[0018] It can be seen from the above technical solutions that in the embodiments of the present application, a large model can be trained based on the target question-answer pair so that the large model supports the question-answer function (i.e. outputs answers to questions). In this way, after obtaining the input question, the large model can be used to obtain the answer to the input question, and the answer to the question can be returned to the user. Only a small amount of computing resources are needed to obtain the answer to the question, and the time to obtain the answer to the question is shorter, thereby saving the computing resources of the server and saving time expenses, and using the analysis capabilities of the large model to give accurate and reliable answers to the question. When obtaining the target question-answer pair, the diversity of the question-answer pair is ensured, and the generation quality of the question-answer pair is improved by introducing semantic fluency to complete the automated production of the question-answer pair. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of a method for obtaining question and answer pairs in one embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of generating question and answer data based on a large model in one embodiment of the present application;

[0021] Figure 3 It is a flowchart of a method for obtaining question and answer pairs in one embodiment of the present application;

[0022] Figure 4 is a schematic diagram of the original table input data in one embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a visualized web page in one embodiment of the present application;

[0024] Figure 6 It is a flowchart of a method for obtaining question and answer pairs in one embodiment of the present application;

[0025] Figure 7 is a structural schematic diagram of a question-answer pair acquisition device in one embodiment of the present application;

[0026] Figure 8 It is a hardware structure diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0027] In the embodiment of the present application, a method for obtaining a question-answer pair is proposed, which can be applied to electronic devices, such as servers, etc. Figure 1 FIG. 1 is a flow chart of the method for obtaining question-answer pairs, which may include:

[0028] Step 101, obtaining original input data; if the original input data includes original table input data, then for each candidate cell in the original table input data, obtaining the row title, column title and cell content corresponding to the candidate cell from the original table input data.

[0029] Step 102, generating a first question and a second question corresponding to the candidate cell based on the row header and the column header; wherein, in the first question, the row header may be located in front of the column header, and in the second question, the column header may be located in front of the row header.

[0030] Step 103: determine the reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question; wherein, if the semantic coherence of the first question is greater than the semantic coherence of the second question, then the reference question may be the first question; if the semantic coherence of the first question is less than the semantic coherence of the second question, then the reference question may be the second question.

[0031] Step 104: Generate an extended question-answer pair based on the reference question and the cell content; wherein the question of the extended question-answer pair is the reference question, and the answer to the extended question-answer pair is the cell content.

[0032] Step 105: Generate a target question-answer pair corresponding to the original input data based on the extended question-answer pair. For example, the target question-answer pair corresponding to the original input data is obtained by verifying the extended question-answer pair.

[0033] Exemplarily, for step 103, before determining the reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question, the following method may be used to determine the semantic coherence of the first question and the semantic coherence of the second question:

[0034] Determine the first syntactic dependency and the first part of speech of each first word in the first question; count the first number of matches between the first syntactic dependency and the first part of speech; determine a first score based on the first number and the total number of first words; determine the first score as the semantic coherence of the first question; determine the second syntactic dependency and the second part of speech of each second word in the second question; count the second number of matches between the second syntactic dependency and the second part of speech; determine a second score based on the second number and the total number of second words; determine the second score as the semantic coherence of the second question. Or,

[0035] Input the first question and the second question into the semantic coherence detection model, so that the semantic coherence detection model outputs a third score corresponding to the first question and a fourth score corresponding to the second question; determine the third score as the semantic coherence of the first question, and determine the fourth score as the semantic coherence of the second question. Or,

[0036] Determine the first syntactic dependency and the first part of speech of each first word in the first question; count the first number of matches between the first syntactic dependency and the first part of speech; determine a first score based on the first number and the total number of first words; determine the second syntactic dependency and the second part of speech of each second word in the second question; count the second number of matches between the second syntactic dependency and the second part of speech; determine a second score based on the second number and the total number of second words; input the first question and the second question into a trained semantic fluency detection model so that the semantic fluency detection model outputs a third score corresponding to the first question and a fourth score corresponding to the second question; perform a weighted operation on the first score and the third score, and determine the score value after the weighted operation as the semantic fluency of the first question, and perform a weighted operation on the second score and the fourth score, and determine the score value after the weighted operation as the semantic fluency of the second question.

[0037] Exemplarily, before performing weighted operations on the first score and the third score, determining the score value after the weighted operation as the semantic coherence of the first question, and performing weighted operations on the second score and the fourth score, and determining the score value after the weighted operation as the semantic coherence of the second question, the method may further include: determining the quotient of the first value and the second value as the weighting coefficient of the first score, and determining the difference between the fixed value and the weighting coefficient of the first score as the weighting coefficient of the third score; determining the quotient of the first value and the second value as the weighting coefficient of the second score, and determining the difference between the fixed value and the weighting coefficient of the second score as the weighting coefficient of the fourth score. The first value is the absolute value of the difference between the first score and the second score; the second value is the sum of the first value and the third value; and the third value is the absolute value of the difference between the third score and the fourth score.

[0038] Exemplarily, if the original input data includes original table input data, the original table input data and the first category of prompt words can also be input into the big model, and the first category of prompt words can be used to guide the big model to provide question-answer pairs based on the original table input data; the big model generates the original question-answer pairs corresponding to the original table input data based on the first category of prompt words and the original table input data, and outputs the original question-answer pairs.

[0039] Generating a target question-answer pair corresponding to the original input data based on the extended question-answer pair may include, but is not limited to: verifying the original question-answer pair and the extended question-answer pair to obtain the target question-answer pair.

[0040] Exemplarily, if the original input data includes original non-tabular input data, it may also include: obtaining a key-value pair based on the original non-tabular input data, and obtaining a target question template corresponding to the key-value pair; wherein the key-value pair may include an attribute name and attribute value information corresponding to the attribute name; generating an extended question-and-answer pair based on the target question template, the attribute name and the attribute value information; wherein the question of the extended question-and-answer pair is determined based on the target question template and the attribute name, and the answer of the extended question-and-answer pair is determined based on the attribute value information; generating a target question-and-answer pair corresponding to the original input data based on the extended question-and-answer pair.

[0041] Exemplarily, obtaining a target question template corresponding to a key-value pair may include, but is not limited to: determining a target attribute type corresponding to the key-value pair, which target attribute type may be an amount type, a time type, a name type, or a place name type; randomly selecting a question template from a plurality of configured question templates corresponding to the target attribute type, and determining the randomly selected question template as the target question template.

[0042] An extended question-answer pair is generated based on the target question template, the attribute name and the attribute value information, which may include but is not limited to: if the target question template includes a first field related to the target attribute type and a second field unrelated to the target attribute type, the question of the extended question-answer pair can be obtained by replacing the first field in the target question template with the attribute name; and the answer to the extended question-answer pair is determined based on the attribute value information.

[0043] Exemplarily, generating an extended question-answer pair based on the target question template, the attribute name, and the attribute value information may include, but is not limited to: if the target question template is a numerical comparison template, and the numerical comparison template includes two fields to be compared, then the two fields to be compared are replaced based on the attribute name and the attribute name in another key-value pair to obtain the question of the extended question-answer pair; the answer to the extended question-answer pair is determined based on the comparison result between the attribute value information and the attribute value information in another key-value pair. Alternatively, if the numerical comparison template includes a field to be compared and a reference value, then the field to be compared is replaced based on the attribute name to obtain the question of the extended question-answer pair; the answer to the extended question-answer pair is determined based on the comparison result between the attribute value information and the reference value. Alternatively, if the target question template is a numerical confirmation template, and the numerical confirmation template includes a field to be compared and a reference value, then the field to be compared in the numerical confirmation template is replaced based on the attribute name to obtain the question of the extended question-answer pair; the answer to the extended question-answer pair is determined based on the comparison result between the attribute value information and the reference value. Alternatively, if the target question template is a maximum value comparison type template, and the maximum value comparison type template includes a reference value, the question of the extended question and answer pair is determined based on the target question template; and the answer to the extended question and answer pair is determined based on the attribute value information and reference values ​​in all key-value pairs.

[0044] Exemplarily, obtaining key-value pairs based on original non-tabular input data may include, but is not limited to: inputting the original non-tabular input data and the second type of prompt words into the large model, the second type of prompt words can be used to guide the large model to provide key-value pairs based on the original non-tabular input data; generating key-value pairs corresponding to the original non-tabular input data based on the second type of prompt words and the original non-tabular input data by the large model, and outputting the key-value pairs. In addition, the first type of prompt words may also be input into the large model, and the large model may generate original question-answer pairs corresponding to the original non-tabular input data based on the first type of prompt words and the original non-tabular input data, and output the original question-answer pairs. Generating a target question-answer pair corresponding to the original input data based on the extended question-answer pair may include: verifying the original question-answer pair and the extended question-answer pair to obtain the target question-answer pair.

[0045] It can be seen from the above technical solutions that in the embodiments of the present application, a large model can be trained based on the target question-answer pair so that the large model supports the question-answer function (i.e. outputs answers to questions). In this way, after obtaining the input question, the large model can be used to obtain the answer to the input question, and the answer to the question can be returned to the user. Only a small amount of computing resources are needed to obtain the answer to the question, and the time to obtain the answer to the question is shorter, thereby saving the computing resources of the server and saving time expenses, and using the analysis capabilities of the large model to give accurate and reliable answers to the question. When obtaining the target question-answer pair, the diversity of the question-answer pair is ensured, and the generation quality of the question-answer pair is improved by introducing semantic fluency to complete the automated production of the question-answer pair.

[0046] The above technical solutions of the embodiments of the present application are described below in combination with specific application scenarios.

[0047] In order to retrieve the data required by the user from a large amount of network data, the user can provide an original input question, and the server retrieves the data required by the user from a large amount of network data based on the original input question and returns the data to the user. However, due to the large amount of network data, the server needs to consume a lot of computing resources to retrieve the data required by the user from a large amount of network data, and the server needs to spend a lot of time, resulting in a large amount of server resource consumption, and the user can only obtain the required data after a long time.

[0048] In view of the above findings, we can use large models (such as large language models, Large Language Model, LLM) to analyze the data. For example, with the help of large models, we can obtain the answer to the input question and return the answer to the user. It only takes a small amount of computing resources to obtain the answer, and the time to obtain the answer is shorter, thus saving server computing resources and time.

[0049] In order to enable the big model to support the question-answering function, it is necessary to first obtain a large number of target question-answer pairs and train the big model based on these target question-answer pairs. The following is a detailed description of the process of obtaining the target question-answer pairs.

[0050] See also Figure 2 As shown, it is a schematic diagram of question and answer data generation based on a large model (such as a large language model or other types of large models). The process may include: prompt word construction, attribute information acquisition, attribute information integration, question and answer pair expansion and question and answer pair verification.

[0051] Exemplarily, the original input data may be document-based input data (referred to as original document input data, such as articles, etc.), image-based input data (referred to as original image input data), table-based input data (referred to as original table input data), and text-based input data (referred to as original text input data). The above are just a few examples of original input data, and there is no restriction on the type of original input data. In this embodiment, the restrictions on scene fields and data types are broken, and data of any application scene can be used as original input data, and data of any data type can be used as original input data.

[0052] Exemplarily, all original input data can be divided into original form input data and original non-form input data. Original form input data is input data based on a form. Original non-form input data is input data of other types except for a form, such as original document input data (i.e., document data), original image input data (i.e., image data), and original text input data (i.e., text data).

[0053] For the original form input data, a question-answer pair acquisition method is proposed in the embodiment of the present application, which can be applied to any type of electronic equipment, such as servers, personal computers, laptops, smart terminals, front-end devices (such as IPCs, etc.), storage devices, etc., without limitation. The question-answer pair acquisition method can generate target question-answer pairs based on the original form input data. Combined with the processes of prompt word construction, attribute information acquisition, attribute information integration, question-answer pair expansion and question-answer pair verification, the generation process of the target question-answer pair is explained. See Figure 3 FIG. 1 is a flow chart of the method for obtaining question-answer pairs, which may include:

[0054] Step 301: obtain original form input data and obtain a first type of prompt words, where the first type of prompt words are used to guide the large model to provide question-answer pairs based on the original form input data.

[0055] For example, the original table input data is table-based input data, and the original table input data is used to construct question-answer pairs, which can be understood as sample data used to construct question-answer pairs.

[0056] For example, in the process of constructing prompt words, the first type of prompt words can be constructed. The first type of prompt words can also be called prompt. The first type of prompt words are instructions input to the large model together with the original table input data, and are used to guide the large model to output relevant information. In order to guide the large model to provide question-answer pairs, the first type of prompt words are used to guide the large model to provide question-answer pairs according to the original table input data.

[0057] For example, the first type of prompt words can be: Please ask three questions and answer them based on the data input in the original table. The specific content of the questions is not limited. Obviously, the main purpose of the first type of prompt words is to guide the large model to output question and answer pairs based on the data input in the original table. The above are just examples of the first type of prompt words, and there is no limitation on this first type of prompt words. In the above example, taking guiding the large model to output three question and answer pairs as an example, the large model can also be guided to output two question and answer pairs, four question and answer pairs, etc., and there is no limit on this number.

[0058] Step 302: input the original table input data and the first category prompt words into the large model, generate original question-answer pairs based on the first category prompt words and the original table input data through the large model, and output the original question-answer pairs.

[0059] Exemplarily, the large model has the function of providing question-answer pairs. There is no restriction on the network structure of the large model and the training process of the large model, as long as the large model has the function of providing question-answer pairs.

[0060] Exemplarily, after the original table input data and the first category of prompt words are input into the big model, since the first category of prompt words are used to guide the big model to provide question-answer pairs according to the original table input data, and the big model has the function of providing question-answer pairs, the big model can generate question-answer pairs based on the original table input data. For the sake of convenience in distinction, the question-answer pair is called the original question-answer pair, and the original question-answer pair is output.

[0061] For example, the large model can find questions from the original table input data or generate questions based on the original table input data. The large model can find answers that match the questions from the original table input data or generate answers that match the questions based on the original table input data. In this way, the question and the answer can be combined into an original question-answer pair, which can also be called a QA pair. The Q (question) in the QA pair can represent the question, and the A (answer) in the QA pair can represent the answer.

[0062] For example, the question of the original question-answer pair may be "What animal performances do tourists watch?", and the answer to the original question-answer pair may be "sea lions". For another example, the question of the original question-answer pair may be "What are the characteristics of the chrysanthemums on the roadside?", and the answer to the original question-answer pair may be "in various shapes and forms".

[0063] Step 303: For each candidate cell in the original table input data, obtain the row title, column title and cell content corresponding to the candidate cell from the original table input data.

[0064] For example, all cells in the original table input data can be used as candidate cells. Alternatively, some cells in the original table input data can be used as candidate cells; for example, several cells can be randomly selected as candidate cells, or several cells in specified positions can be selected as candidate cells, such as the cell in the first row and fourth column as a candidate cell, the cell in the second row and fourth column as a candidate cell, the cell in the second row and seventh column as a candidate cell, and so on.

[0065] For each candidate cell in the original table input data, the row title corresponding to the candidate cell, the column title corresponding to the candidate cell, and the cell content corresponding to the candidate cell can be obtained from the original table input data. For example, based on the relationship between the cell and the row title and the column title, the row title and the column title corresponding to the candidate cell can be found. In addition, the content of the candidate cell can be used as the cell content, and the cell content refers to the data or information stored or displayed in the candidate cell.

[0066] For example, see Figure 4 As shown, it is a schematic diagram of an example of original table input data. When the cell in the first row and fifth column is taken as a candidate cell, the row title corresponding to the candidate cell obtained from the original table input data is "Small Household Appliance Manufacturing Industry", the column title corresponding to the candidate cell obtained from the original table input data is "Increase or decrease in operating costs compared with the same period last year (%)", and the cell content corresponding to the candidate cell obtained from the original table input data is "7.08".

[0067] Exemplarily, the original table input data can be input into the large model, which determines the row headers, column headers, and cell contents corresponding to the candidate cells based on the original table input data, and outputs the row headers, column headers, and cell contents corresponding to the candidate cells. In this way, the row headers, column headers, and cell contents corresponding to the candidate cells can be obtained from the original table input data.

[0068] For example, the big model has the function of obtaining row headers, column headers and cell contents. Based on this, after the original table input data is input into the big model, the big model can output the row headers, column headers and cell contents corresponding to the candidate cells. There is no restriction on the processing process of the big model.

[0069] For example, considering that the large model may have a deviation in understanding the original table input data, the large model may not be able to obtain the row headers, column headers and cell contents corresponding to the candidate cells from the original table input data. Based on this, a table understanding expert model can be pre-trained, and the table understanding expert model is used to obtain the row headers, column headers and cell contents corresponding to the candidate cells from the original table input data.

[0070] For example, the table understanding expert model can be a deep learning model, or a neural network model. There is no restriction on the network structure of the table understanding expert model, and there is no restriction on the training process of the table understanding expert model, as long as the table understanding expert model can obtain the row header, column header, and cell content corresponding to the candidate cell from the original table input data.

[0071] Since the table understanding expert model is used to obtain row headers, column headers and cell contents, after the original table input data is input to the table understanding expert model, the table understanding expert model can output the row headers, column headers and cell contents corresponding to the candidate cells based on the original table input data, and there is no restriction on the processing process of the table understanding expert model. On this basis, the row headers, column headers and cell contents corresponding to the candidate cells can be obtained from the original table input data.

[0072] Exemplarily, the row header, column header, and cell content corresponding to the candidate cell can be referred to as attribute information. In the process of acquiring the attribute information, the original table input data can be input into the table understanding expert model, and the table understanding expert model can acquire the relationship between cells, and the table understanding expert model can acquire the row header and column header corresponding to the cell. Therefore, the table understanding expert model outputs the attribute information, and the attribute information includes the row header, column header, and cell content corresponding to the candidate cell. In this way, in the process of acquiring the attribute information, for each candidate cell in the original table input data, the row header, column header, and cell content corresponding to the candidate cell can be acquired from the original table input data.

[0073] Step 304: For each candidate cell, generate a first question and a second question corresponding to the candidate cell based on the row header and column header corresponding to the candidate cell; wherein, in the first question, the row header is located in front of the column header, and in the second question, the column header is located in front of the row header.

[0074] For example, the first question can be, "row title + of + column title + how much", that is, the row title is located in front of the column title, and the second question can be, "column title + of + row title + how much", that is, the column title is located in front of the row title. Of course, the above are just examples of the first and second questions, as long as "the row title in the first question is located in front of the column title, and the column title in the second question is located in front of the row title".

[0075] For example, see Figure 4 As shown, the row title can be "Small household appliance manufacturing industry" and the column title can be "Increase or decrease in operating costs compared with the same period last year (%)". Based on the row title and column title corresponding to the candidate cell, the first question can be, how much the operating costs of the small household appliance manufacturing industry have increased or decreased compared with the same period last year, and the second question can be, how much the operating costs of the small household appliance manufacturing industry have increased or decreased compared with the same period last year.

[0076] Step 305: Determine the semantic coherence of the first question and the semantic coherence of the second question.

[0077] Exemplarily, the semantic coherence of the first question refers to the degree of semantic coherence. The greater the semantic coherence, the more coherent the semantics of the first question, that is, the greater the possibility that the first question is a complete sentence. The smaller the semantic coherence, the less coherent the semantics of the first question, that is, the less likely the first question is a complete sentence. The semantic coherence of the second question refers to the degree of semantic coherence. The greater the semantic coherence, the more coherent the semantics of the second question, that is, the greater the possibility that the second question is a complete sentence. The smaller the semantic coherence, the less coherent the semantics of the second question, that is, the less likely the second question is a complete sentence. For example, the semantic coherence of the first question and the semantic coherence of the second question can be determined in the following manner:

[0078] Method 1: Determine the syntactic dependency and part of speech of each word, and determine the semantic coherence of the first question and the semantic coherence of the second question based on the syntactic dependency and part of speech of each word. For example, the following steps can be used to determine the semantic coherence of the first question and the semantic coherence of the second question.

[0079] Step S11: divide the first question into multiple first words, and divide the second question into multiple second words. For example, the first question can be divided into multiple first words, and the second question can be divided into multiple second words by word segmentation technology, or other technologies can be used to complete word segmentation, which is not limited.

[0080] Step S12: Determine the first syntactic dependency and the first part of speech of each first word in the first question.

[0081] For example, for each first word in the first question, the first syntactic dependency relationship of the first word and the first part of speech of the first word can be determined. The first syntactic dependency relationship can be a subject-predicate relationship, a verb-object relationship, a subject-predicate relationship, an adverbial relationship, a verb-complement relationship, a parallel relationship, a preposition-object relationship, a subject-predicate-object relationship, etc., without limitation. The first part of speech can be a noun, a verb, an adjective, a numeral, a quantifier, a pronoun, an adverb, a preposition, a conjunction, an auxiliary word, an interjection, an onomatopoeia, etc., without limitation.

[0082] Step S13: Count the first number of matches between the first syntactic dependency and the first part of speech.

[0083] For example, for each first word in the first question, if the first syntactic dependency of the first word matches the first part of speech of the first word, that is, the first syntactic dependency matches the first part of speech correctly, then the value of the first quantity is increased by 1, and the initial value of the first quantity is 0, and 1 is increased each time it is correctly matched. If the first syntactic dependency of the first word does not match the first part of speech of the first word, that is, the first syntactic dependency matches the first part of speech incorrectly, then the value of the first quantity remains unchanged. After performing the above processing on each first word in the first question, the first quantity of correct matches can be obtained.

[0084] The first syntactic dependency relationship correctly matches the first part of speech, which means that when the first word adopts the first syntactic dependency relationship, the first word is allowed to adopt the first part of speech, and when the first word adopts the first part of speech, the first word is allowed to adopt the first syntactic dependency relationship. The first syntactic dependency relationship incorrectly matches the first part of speech, which means that when the first word adopts the first syntactic dependency relationship, the first word is not allowed to adopt the first part of speech, or when the first word adopts the first part of speech, the first word is not allowed to adopt the first syntactic dependency relationship.

[0085] For example, a matching table of syntactic dependencies and parts of speech may be preconfigured. Based on this matching table, it is possible to know which parts of speech the syntactic dependencies correspond to, and which syntactic dependencies the parts of speech correspond to. On this basis, by querying this matching table, it is possible to know that the first syntactic dependency correctly matches the first part of speech, or, by querying this matching table, it is possible to know that the first syntactic dependency incorrectly matches the first part of speech.

[0086] Step S14: Determine a first score based on the first number and the total number of the first words. For example, the quotient between the first number and the total number of the first words may be determined as the first score.

[0087] For example, the first score can be determined using the following formula (1):

[0088] Formula (1)

[0089] represents the first number of correct matches, Indicates the total number of the first word, Indicates the first score. Assuming that the first question is divided into 10 first words, the first syntactic dependency relations of 6 first words match the first part of speech, and the first syntactic dependency relations of 4 first words do not match the first part of speech, then the first quantity is 6, the total quantity of first words is 10, and the first score is 0.6.

[0090] Step S15: Determine the first score as the semantic coherence of the first question.

[0091] For example, if the first score is 0.6, the semantic coherence of the first question may be 0.6.

[0092] Step S16: Determine the second syntactic dependency and the second part of speech of each second word in the second question.

[0093] Step S17: Count the second number of matches between the second syntactic dependency and the second part of speech.

[0094] Step S18: Determine a second score based on the second number and the total number of second words.

[0095] Step S19: Determine the second score as the semantic coherence of the second question.

[0096] Step S16 to step S19 may refer to step S12 to step S15, and will not be repeated here.

[0097] Method 2: Input the first question and the second question into the semantic fluency detection model, so that the semantic fluency detection model outputs a third score corresponding to the first question and a fourth score corresponding to the second question; determine the third score as the semantic fluency of the first question, and determine the fourth score as the semantic fluency of the second question.

[0098] Exemplarily, the first question and the second question can be input into the large model, and the large model outputs a third score corresponding to the first question and a fourth score corresponding to the second question. In this way, the third score can be determined as the semantic coherence of the first question, and the fourth score can be determined as the semantic coherence of the second question.

[0099] For example, the big model has the function of scoring the semantic coherence of a sentence. Based on this, after the first question and the second question are input into the big model, the big model can output the third score corresponding to the first question and the fourth score corresponding to the second question. There is no restriction on the processing process of the big model.

[0100] Exemplarily, considering that the large model may have deviations in its understanding of the first question and the second question, the large model may not be able to output the accurate third score corresponding to the first question and the accurate fourth score corresponding to the second question. Based on this, a semantic fluency detection model can be pre-trained, and the semantic fluency detection model is used to score the semantic fluency of a sentence, and the score range is 0-1. The larger the semantic fluency score, the less fluent the semantics; the smaller the semantic fluency score, the less fluent the semantics. For example, the semantic fluency detection model can be a deep learning model or a neural network model. There is no restriction on the network structure of the semantic fluency detection model, and there is no restriction on the training process of the semantic fluency detection model, as long as the semantic fluency detection model can score the semantic fluency of a sentence.

[0101] Since the semantic fluency detection model is used to score the semantic fluency of a sentence, after the first question is input into the semantic fluency detection model, the semantic fluency detection model can score the first question and obtain a third score corresponding to the first question, and the third score is between 0-1, and there is no restriction on the processing process of the semantic fluency detection model. After the second question is input into the semantic fluency detection model, the semantic fluency detection model can score the second question and obtain a fourth score corresponding to the second question, and the fourth score is between 0-1. In summary, the semantic fluency detection model can output the third score corresponding to the first question and the fourth score corresponding to the second question. Then, the third score can be determined as the semantic fluency of the first question, and the fourth score can be determined as the semantic fluency of the second question.

[0102] Method 3: Based on the first score and the third score corresponding to the first question, the first score and the third score are weighted, and the score value after the weighted operation is determined as the semantic coherence of the first question. Based on the second score and the fourth score corresponding to the second question, the second score and the fourth score are weighted, and the score value after the weighted operation is determined as the semantic coherence of the second question. For example, the semantic coherence of the first question and the semantic coherence of the second question can be determined in the following manner:

[0103] Step S21: Divide the first question into a plurality of first words, and divide the second question into a plurality of second words.

[0104] Step S22: Determine the first syntactic dependency and the first part of speech of each first word in the first question.

[0105] Step S23: Count the first number of matches between the first syntactic dependency and the first part of speech.

[0106] Step S24: Determine a first score based on the first number and the total number of first words.

[0107] Step S25: Determine the second syntactic dependency and the second part of speech of each second word in the second question.

[0108] Step S26: Count the second number of matches between the second syntactic dependency and the second part of speech.

[0109] Step S27: Determine a second score based on the second number and the total number of second words.

[0110] Exemplarily, steps S21 to S27 may refer to method 1, and will not be repeated here.

[0111] Step S28: input the first question and the second question into the semantic fluency detection model, so that the semantic fluency detection model outputs a third score corresponding to the first question and a fourth score corresponding to the second question.

[0112] Exemplarily, step S28 may refer to method 2, which will not be repeated here.

[0113] Step S29: perform a weighted operation on the first score and the third score, and determine the weighted score value as the semantic coherence of the first question. Also, perform a weighted operation on the second score and the fourth score, and determine the weighted score value as the semantic coherence of the second question.

[0114] For example, based on the first score and the third score, the semantic coherence of the first question is determined using formula (2), and based on the second score and the fourth score, the semantic coherence of the second question is determined using formula (3).

[0115] P1= Formula (2)

[0116] P2= Formula (3)

[0117] In formula (2) and formula (3), P1 represents the semantic fluency of the first question. represents the first fraction, represents the weighting coefficient of the first score, represents the third fraction, P2 represents the weighted coefficient of the third score. Obviously, by weighting the first score and the third score, the semantic fluency of the first question is obtained. P2 represents the semantic fluency of the second question. represents the second fraction, represents the weighting coefficient of the second score, represents the fourth fraction, Represents the weighting coefficient of the fourth score. Obviously, by performing weighted operation on the second score and the fourth score, the semantic fluency of the second question is obtained.

[0118] Weighting factor for the first score and the weighting factor of the second score , weighting coefficient and weighting coefficient They can be the same or different, and the following example uses the case where the two are the same.

[0119] Weighting factor for the third score and the weighting factor of the fourth score , weighting coefficient and weighting coefficient They can be the same or different, and the following example uses the case where the two are the same.

[0120] Weighting factor for the first score , weighting coefficient It can be configured based on experience, or the quotient of the first value and the second value can be determined as the weighting coefficient of the first score. For example, the first value may be the absolute value of the difference between the first fraction and the second fraction; the second value may be the sum of the first value and the third value, and the third value may be the absolute value of the difference between the third fraction and the fourth fraction.

[0121] For example, the weighting coefficient of the first score can be determined using the following formula (4): :

[0122] Formula (4)

[0123] In formula (4), It can be the absolute value of the difference between the first score and the second score, that is, the first numerical value. It can be the absolute value of the difference between the third score and the fourth score, that is, the third numerical value. It can be the sum of the first value and the third value, that is, the second value. In summary, the quotient of the first value and the second value can be determined as the weighting coefficient of the first score. .

[0124] The weighting factor in obtaining the first score Then, a fixed value (such as 1) can be added to the weighting coefficient of the first score. The difference between , for example, .

[0125] Weighting factor for the second score , weighting coefficient It can be configured based on experience, or the quotient of the first value and the second value can be determined as the weighting coefficient of the second score. , I will not go into details about this.

[0126] The weighting factor for the second score Then, the fixed value (such as 1) can be added to the weighting coefficient of the second score. The difference is determined as the weighting coefficient of the fourth score , for example, .

[0127] In summary, the weighting coefficient of the first score, the weighting coefficient of the second score, the weighting coefficient of the third score, and the weighting coefficient of the fourth score can be obtained. In combination with the first score, the second score, the third score, and the fourth score, the first score and the third score can be weighted, and the score value after the weighted operation is determined as the semantic coherence of the first question. And, the second score and the fourth score can be weighted, and the score value after the weighted operation is determined as the semantic coherence of the second question.

[0128] For example, by performing a weighted operation on the first score and the third score, the weighted score value is determined as the semantic fluency of the first question, which can assist semantic understanding based on syntactic analysis, and ensure the quality of the generated sentence by integrating syntactic analysis and semantic understanding. By performing a weighted operation on the second score and the fourth score, the weighted score value is determined as the semantic fluency of the second question, which can assist semantic understanding based on syntactic analysis, and ensure the quality of the generated sentence by integrating syntactic analysis and semantic understanding. Considering that the score difference between the two sentences is larger, the score is more reliable, therefore, the weight coefficient of the first score is determined based on the difference in sentence scores as the weight of the corresponding method score, that is, formula (4) is used to determine the weight coefficient of the first score.

[0129] Step 306: Determine the reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question. For example, if the semantic coherence of the first question is greater than the semantic coherence of the second question, the reference question may be the first question; or, if the semantic coherence of the first question is less than the semantic coherence of the second question, the reference question may be the second question.

[0130] Step 307: Generate an extended question-answer pair based on the reference question and the cell content corresponding to the candidate cell, where the question of the extended question-answer pair is the reference question, and the answer to the extended question-answer pair is the cell content.

[0131] For example, if the semantic coherence of the first question is greater than that of the second question, the first question can be used as the reference question corresponding to the candidate cell. Based on this, the question of the expanded question and answer pair is the first question, and the answer to the expanded question and answer pair is the cell content corresponding to the candidate cell.

[0132] Alternatively, if the semantic coherence of the first question is less than that of the second question, the second question can be used as the reference question corresponding to the candidate cell. Based on this, the question of the expanded question and answer pair is the second question, and the answer to the expanded question and answer pair is the cell content corresponding to the candidate cell.

[0133] For example, if the semantic coherence of the first question is equal to the semantic coherence of the second question, the first question can be used as the reference question corresponding to the candidate cell, and the second question can also be used as the reference question corresponding to the candidate cell. The question of the extended question-answer pair can be the first question, and the question of the extended question-answer pair can also be the second question. The answer to the extended question-answer pair is the cell content corresponding to the candidate cell.

[0134] For example, if the semantic coherence of the first question is less than a preset threshold, and the semantic coherence of the second question is also less than a preset threshold, regardless of whether the semantic coherence of the first question is equal to the semantic coherence of the second question, the first question can be used as a reference question corresponding to the candidate cell, and the second question can be used as a reference question corresponding to the candidate cell. The question of the extended question-answer pair can be the first question or the second question, and the answer to the extended question-answer pair is the cell content corresponding to the candidate cell.

[0135] In summary, for each candidate cell in the original table input data, the extended question-answer pair corresponding to the candidate cell can be obtained, thereby obtaining multiple extended question-answer pairs corresponding to the original table input data.

[0136] Exemplarily, the processing of step 304 to step 307 can also be referred to as a question-answer pair expansion process. For example, in the question-answer pair expansion process, for each candidate cell, a first question and a second question corresponding to the candidate cell are generated based on the row header and the column header corresponding to the candidate cell. The semantic coherence of the first question and the semantic coherence of the second question are determined. Based on the semantic coherence of the first question and the semantic coherence of the second question, a reference question corresponding to the candidate cell is determined. An extended question-answer pair is generated based on the reference question and the cell content corresponding to the candidate cell, and there is no restriction on this question-answer pair expansion process.

[0137] Step 308: Verify the original question-answer pair and the expanded question-answer pair to obtain the target question-answer pair corresponding to the original table input data. The target question-answer pair is used to train the large model (i.e., the target question-answer pair is used as sample data to fine-tune the large model), or the target question-answer pair is used to test the large model (i.e., the target question-answer pair is used as test data to test the large model to improve the performance of the large model). In addition to training and testing large models, the target question-answer pair can also train / test other types of models. There is no restriction on the function of the target question-answer pair, i.e., the target question-answer pair can be used as a label data to participate in model training and testing.

[0138] For example, during the question-answer pair verification process, the original question-answer pair can be verified, or the extended question-answer pair can be verified. The verification method for the original question-answer pair and the extended question-answer pair is the same. After the original question-answer pair and the extended question-answer pair are verified, the verified question-answer pair can be used as the target question-answer pair.

[0139] For example, when verifying the original question-answer pair, the original question-answer pair can be directly used as the target question-answer pair, and when verifying the extended question-answer pair, the extended question-answer pair can be directly used as the target question-answer pair. Alternatively, after obtaining a large number of question-answer pairs (QA pairs, such as original question-answer pairs and extended question-answer pairs), considering that these question-answer pairs may not be accurate, these question-answer pairs can also be verified (such as adding, deleting, modifying, checking, etc. on the question-answer pairs), and the verified question-answer pairs can be used as the target question-answer pairs.

[0140] For example, the original question-answer pairs constructed by the large model and the extended question-answer pairs constructed based on attribute information can be added to the question-answer pair database. Then, operations such as adding, deleting, modifying, and viewing can be performed on the question-answer pairs in the question-answer pair database, and the question-answer pairs after the operation can be used as the target question-answer pairs.

[0141] In order to verify the question-answer pairs, a visualization web page can also be designed to display the question-answer pairs in the question-answer database, making it convenient to add, delete, modify, and query the question-answer pairs. Figure 5 The figure shows a schematic diagram of a visualization web page, which may include the following parts.

[0142] Original form input data. The original form input data is displayed through a visual web page, which is convenient for users to view the original form input data and then know whether the question and answer to be verified are accurate and whether they need to be modified.

[0143] OCR (Optical Character Recognition) content. OCR content is obtained by performing OCR operation on the original form input data, which makes it easier for users to understand the original form input data.

[0144] The existing QA pairs (i.e. the original Q&A pairs and the extended Q&A pairs) are displayed on a visual web page, which facilitates users to add, delete, modify, and query these QA pairs. For each QA pair, the visual web page can display Questions, Answers, Providers, Checked, Deleted (Option 1), and Checked / Modified (Option 2).

[0145] For example, Questions can represent questions in a QA pair. Answers can represent answers in a QA pair. Providers can represent the verifier (i.e., the verifier performs operations such as adding, deleting, modifying, and checking QA pairs). Checked can indicate whether it has been checked (e.g., 1 means checked, 0 means unchecked). Based on this, you can check the "Whether Checked" column and perform check operations on QA pairs with a status of 0.

[0146] Option 1 can be used to indicate deletion. When you need to delete a QA pair, you can click Option 1 of the QA pair to delete it. If the user thinks the question is ambiguous, he can directly edit the content of Questions to modify it. Figure 5 The question "What is the ordering party?" in the question is not semantically sound and should be changed to "Who is the ordering party?" If the user thinks the answer is ambiguous, he or she can directly edit the Answers to modify it. Option 2 can indicate verified / modified. After the Questions and Answers are edited, click "Verified / Modified" to automatically save the modified Questions and Answers to the database, and automatically refresh the "Verified" column to change the status to 1.

[0147] Add new OA pairs. New QA pairs can be added through the visualization web page, thereby increasing the difficulty and richness of the QA pairs. For example, new questions can be added, such as "How many products did the ordering party purchase this time?", "What is the price difference between the two products?", and answers to these questions can be given.

[0148] For the original non-tabular input data, a method for obtaining a question-answer pair is proposed in the embodiment of the present application, which can generate a target question-answer pair based on the original non-tabular input data. The process of generating the target question-answer pair is described by combining the processes of prompt word construction, attribute information acquisition, attribute information integration, question-answer pair expansion, and question-answer pair verification. Figure 6 FIG. 1 is a flow chart of a method for obtaining question-answer pairs, and the method may include:

[0149] Step 601, obtain original non-tabular input data, and obtain first-category prompt words and second-category prompt words, wherein the first-category prompt words are used to guide the large model to provide question-answer pairs based on the original non-tabular input data, and the second-category prompt words are used to guide the large model to provide key-value pairs based on the original non-tabular input data.

[0150] For example, the original non-tabular input data may be image-based input data, text-based input data, or document-based input data, and the original non-tabular input data is used to construct question-answer pairs.

[0151] For example, in the process of constructing prompt words, the first category of prompt words and the second category of prompt words are constructed. The content of the first category of prompt words can be found in step 301. The second category of prompt words can also be called Prompt. The second category of prompt words are instructions input to the large model together with the original non-tabular input data. The second category of prompt words are used to guide the large model to provide key-value pairs (i.e., KV pairs) according to the original non-tabular input data.

[0152] For example, the second type of prompt words can be: Please give the key value information in the original non-tabular input data. If a key has multiple values, please give them in the form of a list from top to bottom and from left to right. The specific content of the question is not limited. Obviously, the main purpose of this second type of prompt words is to guide the large model to output key value information (i.e., key-value pairs, which can include Key and Value) for the original non-tabular input data. The above is just an example of the second type of prompt words, and there is no restriction on this second type of prompt words.

[0153] For example, a big model can be a multimodal big model. A multimodal big model is a big model that supports multiple data types, such as images, tables, texts, documents, and other data types.

[0154] Step 602: Input the original non-tabular input data, the first category prompt words, and the second category prompt words into the large model, generate original question-answer pairs based on the first category prompt words and the original non-tabular input data through the large model, and output the original question-answer pairs. Step 602 is similar to step 302 and will not be repeated here.

[0155] Step 603: Obtain key-value pairs based on the original non-tabular input data. For example, the key-value pairs corresponding to the original non-tabular input data are generated based on the second type of prompt words and the original non-tabular input data by the large model, and the key-value pairs are output. In this way, the key-value pairs corresponding to the original non-tabular input data can be obtained.

[0156] Exemplarily, the large model has the function of providing key-value pairs, and there is no restriction on the network structure of the large model and the training process of the large model, as long as the large model has the function of providing key-value pairs. After the original non-tabular input data and the second type of prompt words are input to the large model, since the second type of prompt words are used to guide the large model to provide key-value pairs according to the original non-tabular input data, and the large model has the function of providing key-value pairs, the large model can generate key-value pairs based on the original non-tabular input data and output the key-value pairs.

[0157] The large model can find the key from the original non-tabular input data or automatically generate the key based on the original non-tabular input data. The large model can find the value that matches the key from the original non-tabular input data or automatically generate the value based on the original non-tabular input data. The key and the value can be combined into a key-value pair, which can also be called a KV pair. The K in the KV pair represents the key, and the V in the KV pair represents the value.

[0158] Exemplarily, the key-value pair may be referred to as attribute information. In the process of acquiring the attribute information, the original non-tabular input data may be input to the large model, and the large model outputs the attribute information, and the attribute information includes the key-value pair. In this way, in the process of acquiring the attribute information, the key-value pair is acquired based on the original non-tabular input data.

[0159] For attribute information, the attribute information may include an attribute name and an attribute value. For example, the Key content is the attribute name in the attribute information, and the Value content is the attribute value in the attribute information.

[0160] For example, the Key content of the attribute information may be "product name", the Value content of the attribute information may be "price tag of product in pharmacy and supermarket", the Key content of the attribute information may be "place of origin", the Value content of the attribute information may be "Changsha, Hunan", the Key content of the attribute information may be "specification", the Value content of the attribute information may be "1200 sheets per pack", the Key content of the attribute information may be "unit", the Value content of the attribute information may be "2 packs", the Key content of the attribute information may be "barcode", the Value content of the attribute information may be "18.9 per pack", the Key content of the attribute information may be "retail price", and the Value content of the attribute information may be "18.90".

[0161] Exemplarily, when obtaining key-value pairs based on original non-tabular input data, in addition to using a large model to obtain key-value pairs, an additional expert model can be introduced to extract key-value pairs from the original non-tabular input data through the expert model. For example, for original non-tabular input data of image type, a relational extraction expert model can be used to obtain key-value pairs. For original non-tabular input data of layout type (document type), a layout analysis expert model can be used to obtain key-value pairs, such as using a layout analysis expert model to obtain key-value pairs of figure captions and table annotations, key-value pairs of headers and footers, etc. There is no restriction on this expert model, as long as the expert model can extract key-value pairs from the original non-tabular input data, and its implementation is similar to the large model.

[0162] Step 604: for each key-value pair, determine the target attribute type corresponding to the key-value pair, and the target attribute type may be an amount type, a time type, a person name type, or a place name type.

[0163] Exemplarily, in the process of integrating attribute information, the attribute information of all key-value pairs can be integrated to obtain key-value pairs of different attribute types. For example, all key-value pairs of the amount type can be obtained, that is, the target attribute type corresponding to these key-value pairs is the amount type. All key-value pairs of the time type can be obtained, that is, the target attribute type corresponding to these key-value pairs is the time type. All key-value pairs of the person name type can be obtained, that is, the target attribute type corresponding to these key-value pairs is the person name type. All key-value pairs of the place name type can be obtained, that is, the target attribute type corresponding to these key-value pairs is the place name type. In addition to the amount type, time type, person name type and place name type, there can also be other attribute types, which are not limited to this.

[0164] For example, all key-value pairs can be classified based on the key. For scenarios such as tickets, invoices, and admission tickets, the key in the key-value pair carries attributes such as amount and time, so the amount type and time type can be divided. For scenarios such as mail, newspapers, and test papers, the key in the key-value pair carries attributes such as name and place name, so the name type and place name type can be divided. For the relationship between attribute type and key, see Table 1. The target attribute type corresponding to the key-value pair can be determined by the key in the key-value pair.

[0165] Table 1

[0166]

[0167] In summary, for each key-value pair, the attribute type corresponding to the key-value pair can be obtained through the Key query table 1 in the key-value pair, and the attribute type is used as the target attribute type corresponding to the key-value pair.

[0168] Step 605: For each key-value pair, obtain a target question template corresponding to the key-value pair, and generate an extended question-answer pair based on the target question template and the key-value pair. For example, the key-value pair may include an attribute name (such as the Key in the key-value pair) and attribute value information corresponding to the attribute name (such as the Value in the key-value pair), and the extended question-answer pair may be generated based on the target question template, the attribute name, and the attribute value information.

[0169] For example, in the process of expanding the question-answer pair, for each key-value pair, an extended question-answer pair can be generated based on the attribute name and attribute value information in the key-value pair and the target question template corresponding to the key-value pair.

[0170] In a possible implementation, the following situation may be used to generate an extended question-answer pair:

[0171] Case 1: Using the template-based question expansion method to generate an extended question-answer pair, in case 1, a question template can be randomly selected from multiple configured question templates corresponding to the target attribute type, and the randomly selected question template can be determined as the target question template. Then, an extended question-answer pair is generated based on the target question template, attribute name, and attribute value information; wherein the question of the extended question-answer pair is determined based on the target question template and attribute name, and the answer of the extended question-answer pair is determined based on the attribute value information.

[0172] For example, for the amount type, you can pre-configure multiple question templates of the amount type, such as how much is the amount, how much is the amount of this purchase, how much is spent, how much is the total expenditure, how much is the expenditure of all goods, etc. For the time type, you can pre-configure multiple question templates of the time type, such as **what is the time, **what is the date, what is the time, what is the date.

[0173] For each key-value pair, if the target attribute type corresponding to the key-value pair is the amount type, a question template is randomly selected from multiple question templates corresponding to the amount type, and the randomly selected question template is determined as the target question template. For example, by designing a template-based question expansion method, multiple sentence question templates are designed for each attribute type. For each key-value pair, one of the expressions (i.e., question template) is randomly selected as the question, thereby greatly improving the diversity of questions.

[0174] After obtaining the target question template, if the target question template includes a first field related to the target attribute type and a second field unrelated to the target attribute type, the question of the extended question and answer pair can be obtained by replacing the first field in the target question template with the attribute name, and the answer to the extended question and answer pair can be determined based on the attribute value information.

[0175] For example, if the target question template is "How much is the amount of this purchase", the first field related to the amount type is "amount", and the second field unrelated to the amount type is "this purchase" and how much is it, etc. Then, the first field in the target question template can be replaced by the attribute name in the key-value pair, and the modified target question template can be used as the question of the extended question-answer pair, that is, the question of the extended question-answer pair is determined based on the target question template and the attribute name. In addition, the attribute value information in the key-value pair can be used as the answer to the extended question-answer pair, that is, the answer to the extended question-answer pair is determined based on the attribute value information.

[0176] Case 2: Using the logical calculation question expansion method to generate extended question-answer pairs. In case 2, considering that some scenarios have digital characteristics (such as commodity amount, commodity quantity, etc.), they can be used as mathematical logic calculation question-answer pair designs. Therefore, a numerical comparison template can be designed, and the numerical comparison template includes two fields to be compared. For example, the numerical comparison template can be: Is the unit price of (a certain commodity A) more expensive / cheaper than that of (a certain commodity B), and is the quantity of (a certain commodity A) more / less than that of (a certain commodity B).

[0177] Based on this, if the attribute name in the key-value pair is the product name (or other name, as long as it matches the name to be compared in the numerical comparison template), and the attribute value information in the key-value pair is a feature of a digital type (such as product amount, product quantity, etc.), then the numerical comparison template can be used as the target question template.

[0178] Then, an extended question-answer pair is generated based on the target question template, the attribute name and the attribute value information. The question of the extended question-answer pair is determined based on the target question template and the attribute name, and the answer of the extended question-answer pair is determined based on the attribute value information. For example, the two fields to be compared in the target question template are replaced based on the attribute name in one key-value pair and the attribute name in another key-value pair to obtain the question of the extended question-answer pair. In addition, the answer to the extended question-answer pair is determined based on the comparison result of the attribute value information in one key-value pair and the attribute value information in another key-value pair. For example, the attribute name in one key-value pair is replaced with "a certain commodity A" in the target question template, and the attribute name in another key-value pair is replaced with "a certain commodity B" in the target question template to obtain the question of the extended question-answer pair. If the attribute value information in one key-value pair is greater than the attribute value information in another key-value pair, the answer to the extended question-answer pair is "yes", and if the attribute value information in one key-value pair is less than the attribute value information in another key-value pair, the answer to the extended question-answer pair is "no".

[0179] Case 3: Using the logical calculation question expansion method to generate extended question-answer pairs, in case 3, a numerical comparison template can be designed, and the numerical comparison template includes the field to be compared and the reference value. For example, the numerical comparison template can be: Is the purchase quantity of (a certain product) more / less than (a certain value), is the unit price of (a certain product) more / cheaper than (a certain value), where "a certain value" represents the reference value. For example, the reference value can be adjusted dynamically, and different reference values ​​can be set for different comparison objects.

[0180] Based on this, if the attribute name in the key-value pair is the product name, and the attribute value information in the key-value pair is a feature of a digital type, the numerical comparison type template can be used as the target question template. Then, an extended question and answer pair is generated based on the target question template, the attribute name, and the attribute value information. The question of the extended question and answer pair is determined based on the target question template and the attribute name, and the answer to the extended question and answer pair is determined based on the attribute value information. For example, the field to be compared is replaced based on the attribute name to obtain the question of the extended question and answer pair. In addition, the answer to the extended question and answer pair is determined based on the comparison result of the attribute value information and the reference value.

[0181] For example, replace the attribute name in the key-value pair with "a certain product" in the target question template to obtain the question of the extended question-answer pair. If the attribute value information in the key-value pair is greater than the reference value, the answer to the extended question-answer pair is "yes", and if the attribute value information is less than the reference value, the answer to the extended question-answer pair is "no".

[0182] Case 4: The extended question-answer pair is generated by the question expansion method of logical calculation. In case 4, a numerical confirmation template can be designed, and the numerical confirmation template includes the field to be compared and the reference value. For example, the reference value can be adjusted dynamically, and different reference values ​​can be set for different comparison objects. Based on the numerical confirmation template, it is necessary to do some calculations on the relevant values ​​of the goods during design. By increasing, decreasing or keeping the true value unchanged, the answer is controlled to be yes or no, which is used to examine the model's ability to calculate values. For example, is the unit price of (a certain product) (a certain number), is the quantity of (a certain product) (a certain number), has (a certain product) purchased (a certain value), and has a total of (a number) products been purchased.

[0183] Based on this, if the attribute name in the key-value pair is the product name, and the attribute value information in the key-value pair is a feature of a digital type, the numerical confirmation class template can be used as the target question template. Then, an extended question and answer pair is generated based on the target question template, the attribute name, and the attribute value information. The question of the extended question and answer pair is determined based on the target question template and the attribute name, and the answer to the extended question and answer pair is determined based on the attribute value information. For example, the question of the extended question and answer pair is obtained by replacing the field to be compared based on the attribute name. In addition, based on the comparison result between the attribute value information and the reference value, the answer to the extended question and answer pair is determined.

[0184] Case 5: The extended question-answer pair is generated by using the logical calculation problem expansion method. In case 5, a maximum value comparison template can be designed, and the maximum value comparison template includes a reference value. For example, the reference value can be adjusted dynamically, and different reference values ​​can be set for different comparison objects. Based on the maximum value comparison template, when designing, it is necessary to compare all values ​​to select the maximum value, and control the answer to yes or no by increasing, decreasing or keeping the true value unchanged, so as to examine the model's ability to compare values. For example, is the most expensive more expensive than the cheapest (number), and is the most expensive / cheapest more / less than (number).

[0185] Based on this, if the attribute value information in the key-value pair is a feature of a digital type, the maximum value comparison class template can be used as the target question template. Then, an extended question and answer pair is generated based on the target question template, the attribute name and the attribute value information. The question of the extended question and answer pair is determined based on the target question template, and the answer to the extended question and answer pair is determined based on the attribute value information. For example, the target question template is directly determined as the question of the extended question and answer pair. Based on the attribute value information and reference values ​​in all key-value pairs, the answer to the extended question and answer pair is determined. For example, based on the attribute value information in all key-value pairs, the maximum attribute value information and the minimum attribute value information are selected, and the answer to the extended question and answer pair is determined based on the comparison result of the difference between the two and the reference value.

[0186] Step 606: Verify the original question-answer pair and the expanded question-answer pair to obtain the target question-answer pair corresponding to the original non-tabular input data. The target question-answer pair is used to train the large model, or the target question-answer pair is used to test the large model. The target question-answer pair can be used as a label data to participate in model training and testing.

[0187] Exemplarily, the implementation process of step 606 can refer to step 308, which will not be repeated here.

[0188] It can be seen from the above technical solutions that in the embodiments of the present application, when obtaining the target question-answer pairs, the diversity of the question-answer pairs is ensured, and the generation quality of the question-answer pairs is improved by introducing semantic fluency to complete the automated production of the question-answer pairs. It is able to break through the format, scene, and field restrictions of documents or images, complete the automated production of question-answer pairs, realize the generation of diverse question-answer pairs through attribute integration and template design, and expand different question-asking methods based on different attributes to ensure the diversity of question-answer pairs. A question-answer pair generation scheme is designed for table scenarios, a table understanding expert model is used to obtain structural information, and a sentence semantic fluency scoring algorithm is designed. The algorithm integrates the model evaluation score to effectively improve the quality of question design. A question-answer pair visualization verification webpage is designed to further improve the diversity and quality of question-answer pair generation, and effectively reduce the cost of question-answer pair verification.

[0189] Based on the same application concept as the above method, a question-answer pair acquisition device is proposed in the embodiment of the present application, see Figure 7 FIG. 1 is a schematic diagram of the structure of the device, and the device may include:

[0190] The acquisition module 71 is used to acquire original input data; if the original input data includes original table input data, for each candidate cell in the original table input data, the row title, column title and cell content corresponding to the candidate cell are acquired from the original table input data;

[0191] A generating module 72, configured to generate a first question and a second question corresponding to the candidate cell based on the row header and the column header; wherein, in the first question, the row header is located before the column header, and in the second question, the column header is located before the row header;

[0192] A determination module 73 is used to determine a reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question; wherein, if the semantic coherence of the first question is greater than the semantic coherence of the second question, the reference question is the first question; if the semantic coherence of the first question is less than the semantic coherence of the second question, the reference question is the second question;

[0193] Processing module 74 is used to generate an extended question-answer pair based on the reference question and the cell content; wherein the question of the extended question-answer pair is the reference question, and the answer to the extended question-answer pair is the cell content; and generate a target question-answer pair corresponding to the original input data based on the extended question-answer pair.

[0194] Exemplarily, the determination module 73 is further used to determine the first syntactic dependency and the first part of speech of each first word in the first question; count the first number of matches between the first syntactic dependency and the first part of speech; determine a first score based on the first number and the total number of first words; determine the first score as the semantic coherence of the first question; determine the second syntactic dependency and the second part of speech of each second word in the second question; count the second number of matches between the second syntactic dependency and the second part of speech; determine a second score based on the second number and the total number of second words; determine the second score as the semantic coherence of the second question; or or, inputting the first question and the second question into a trained semantic fluency detection model so that the semantic fluency detection model outputs a third score corresponding to the first question and a fourth score corresponding to the second question; determining the third score as the semantic fluency of the first question, and determining the fourth score as the semantic fluency of the second question; or, performing a weighted operation on the first score and the third score, determining the weighted score value as the semantic fluency of the first question, performing a weighted operation on the second score and the fourth score, and determining the weighted score value as the semantic fluency of the second question.

[0195] Exemplarily, the determination module 73 is also used to determine the quotient of the first numerical value and the second numerical value as the weighting coefficient of the first score, and determine the difference between the fixed numerical value and the weighting coefficient of the first score as the weighting coefficient of the third score; determine the quotient of the first numerical value and the second numerical value as the weighting coefficient of the second score, and determine the difference between the fixed numerical value and the weighting coefficient of the second score as the weighting coefficient of the fourth score; wherein the first numerical value is the absolute value of the difference between the first score and the second score; the second numerical value is the sum of the first numerical value and the third numerical value; and the third numerical value is the absolute value of the difference between the third score and the fourth score.

[0196] Exemplarily, if the original input data includes original table input data, the generating module 72 is further used to input the original table input data and the first type of prompt words into the large model, the first type of prompt words being used to guide the large model to provide question-answer pairs according to the original table input data; the large model generates the original question-answer pairs corresponding to the original table input data based on the first type of prompt words and the original table input data, and outputs the original question-answer pairs; when the processing module 74 generates the target question-answer pairs corresponding to the original input data based on the extended question-answer pairs, it is specifically used to: verify the original question-answer pairs and the extended question-answer pairs to obtain the target question-answer pairs.

[0197] Exemplarily, if the original input data includes original non-tabular input data, the acquisition module 71 is further used to obtain a key-value pair based on the original non-tabular input data, and obtain a target question template corresponding to the key-value pair; the key-value pair includes an attribute name and attribute value information corresponding to the attribute name; the generation module 72 is further used to generate an extended question-answer pair based on the target question template, the attribute name and the attribute value information; the question of the extended question-answer pair is determined based on the target question template and the attribute name, and the answer of the extended question-answer pair is determined based on the attribute value information; the processing module 74 is further used to generate a target question-answer pair corresponding to the original input data based on the extended question-answer pair.

[0198] Exemplarily, when the acquisition module 71 acquires the target question template corresponding to the key-value pair, it is specifically used to: determine the target attribute type corresponding to the key-value pair, where the target attribute type is an amount type, a time type, a name type, or a place name type; randomly select a question template from multiple configured question templates corresponding to the target attribute type, and determine the randomly selected question template as the target question template; when the generation module 72 generates an extended question and answer pair based on the target question template, the attribute name and the attribute value information, it is specifically used to: if the target question template includes a first field related to the target attribute type and a second field unrelated to the target attribute type, then replace the first field with the attribute name to obtain the question of the extended question and answer pair; and determine the answer to the extended question and answer pair based on the attribute value information.

[0199] Exemplarily, when the generation module 72 generates an extended question-answer pair based on the target question template, the attribute name and the attribute value information, it is specifically used to: if the target question template is a numerical comparison template, and the numerical comparison template includes two fields to be compared, then the two fields to be compared are replaced based on the attribute name and the attribute name in another key-value pair to obtain the question of the extended question-answer pair; based on the comparison result of the attribute value information and the attribute value information in the other key-value pair, the answer to the extended question-answer pair is determined; or, if the numerical comparison template includes fields to be compared and reference values, then the fields to be compared are replaced based on the attribute name to obtain the question of the extended question-answer pair; based on the attribute value information and The comparison result of the reference numerical value determines the answer to the extended question and answer pair; or, if the target question template is a numerical confirmation type template, and the numerical confirmation type template includes a field to be compared and a reference numerical value, then the field to be compared in the numerical confirmation type template is replaced based on the attribute name to obtain the question of the extended question and answer pair; based on the comparison result between the attribute value information and the reference numerical value, the answer to the extended question and answer pair is determined; or, if the target question template is a maximum value comparison type template, and the maximum value comparison type template includes a reference numerical value, then the question of the extended question and answer pair is determined based on the target question template; based on the attribute value information in all key-value pairs and the reference numerical value, the answer to the extended question and answer pair is determined.

[0200] Exemplarily, when the acquisition module 71 acquires key-value pairs based on the original non-tabular input data, it is specifically used to: input the original non-tabular input data and the second type of prompt words into the large model, the second type of prompt words are used to guide the large model to provide key-value pairs according to the original non-tabular input data; generate key-value pairs corresponding to the original non-tabular input data based on the second type of prompt words and the original non-tabular input data through the large model, and output the key-value pairs; the generation module 72 is also used to input the first type of prompt words into the large model, generate original question-answer pairs corresponding to the original non-tabular input data based on the first type of prompt words and the original non-tabular input data through the large model, and output the original question-answer pairs; when the processing module 74 generates a target question-answer pair corresponding to the original input data based on the extended question-answer pair, it is specifically used to: verify the original question-answer pair and the extended question-answer pair to obtain a target question-answer pair.

[0201] Based on the same application concept as the above method, an electronic device is proposed in the embodiment of the present application, see Figure 8 As shown, it includes: a processor 81 and a machine-readable storage medium 82, the machine-readable storage medium 82 stores machine-executable instructions that can be executed by the processor 81; the processor 81 is used to execute the machine-executable instructions to implement the question-answer pair acquisition method disclosed in the above example of this application.

[0202] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the question-and-answer pair acquisition method disclosed in the above example of the present application can be implemented.

[0203] The above-mentioned machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0204] Based on the same application concept as the above method, an embodiment of the present application also provides a computer program product, which may include a computer program. When the computer program is executed by a processor, it implements the question-answer pair acquisition method disclosed in the above example of the present application.

[0205] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0206] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for obtaining question-answer pairs, characterized in that: The method comprises: Acquire original input data; if the original input data includes original table input data, for each candidate cell in the original table input data, acquire the row title, column title and cell content corresponding to the candidate cell from the original table input data; Generate a first question and a second question corresponding to the candidate cell based on the row header and the column header; wherein, in the first question, the row header is located in front of the column header, and in the second question, the column header is located in front of the row header; wherein, the first question is, what is the row header + the column header +, and the second question is, what is the column header + the row header +; Determine a reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question; wherein, if the semantic coherence of the first question is greater than the semantic coherence of the second question, the reference question is the first question; if the semantic coherence of the first question is less than the semantic coherence of the second question, the reference question is the second question; Generate an extended question-answer pair based on the reference question and the cell content; wherein the question of the extended question-answer pair is the reference question, and the answer to the extended question-answer pair is the cell content; A target question-answer pair corresponding to the original input data is generated based on the expanded question-answer pair.

2. The method according to claim 1, characterized in that: Before determining the reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question, the method further includes: Determine a first syntactic dependency relationship and a first part of speech for each first word in the first question; count a first number of matches between the first syntactic dependency relationship and the first part of speech; determine a first score based on the first number and the total number of first words; determine the first score as the semantic coherence of the first question; Determine a second syntactic dependency relationship and a second part of speech for each second word in the second question; count a second number of matches between the second syntactic dependency relationship and the second part of speech; determine a second score based on the second number and the total number of second words; determine the second score as the semantic coherence of the second question; Alternatively, the first question and the second question are input into a trained semantic fluency detection model, so that the semantic fluency detection model outputs a third score corresponding to the first question and a fourth score corresponding to the second question; the third score is determined as the semantic fluency of the first question, and the fourth score is determined as the semantic fluency of the second question; Alternatively, a weighted operation is performed on the first score and the third score, and the score value after the weighted operation is determined as the semantic coherence of the first question; a weighted operation is performed on the second score and the fourth score, and the score value after the weighted operation is determined as the semantic coherence of the second question.

3. The method according to claim 2, characterized in that The method further comprises: Determining a quotient of a first numerical value and a second numerical value as a weighting coefficient of the first score, and determining a difference between a fixed numerical value and the weighting coefficient of the first score as a weighting coefficient of the third score; Determining a quotient of the first numerical value and the second numerical value as a weighting coefficient of the second score, and determining a difference between the fixed numerical value and the weighting coefficient of the second score as a weighting coefficient of the fourth score; wherein the first value is the absolute value of the difference between the first score and the second score; The second value is the sum of the first value and the third value; The third numerical value is an absolute value of a difference between the third score and the fourth score.

4. The method according to claim 1, characterized in that: If the original input data includes original table input data, the method further includes: Inputting the original table input data and the first type of prompt words into the large model, wherein the first type of prompt words are used to guide the large model to provide question-answer pairs according to the original table input data; Generate an original question-answer pair corresponding to the original table input data based on the first category of prompt words and the original table input data through the large model, and output the original question-answer pair; The generating a target question-answer pair corresponding to the original input data based on the extended question-answer pair includes: verifying the original question-answer pair and the extended question-answer pair to obtain a target question-answer pair.

5. The method according to any one of claims 1 to 4, characterized in that: If the original input data includes original non-tabular input data, the method further includes: Acquire a key-value pair based on the original non-tabular input data, and acquire a target question template corresponding to the key-value pair; wherein the key-value pair includes an attribute name and attribute value information corresponding to the attribute name; Generate an extended question-answer pair based on the target question template, the attribute name, and the attribute value information; wherein the question of the extended question-answer pair is determined based on the target question template and the attribute name, and the answer to the extended question-answer pair is determined based on the attribute value information; A target question-answer pair corresponding to the original input data is generated based on the expanded question-answer pair.

6. The method according to claim 5, characterized in that The obtaining of the target question template corresponding to the key-value pair includes: determining a target attribute type corresponding to the key-value pair, wherein the target attribute type is an amount type, a time type, a person name type, or a place name type; randomly selecting a question template from a plurality of configured question templates corresponding to the target attribute type, and determining the randomly selected question template as the target question template; The generating of an extended question-answer pair based on the target question template, the attribute name and the attribute value information includes: if the target question template includes a first field related to a target attribute type and a second field unrelated to the target attribute type, obtaining the question of the extended question-answer pair by replacing the first field with the attribute name; and determining the answer to the extended question-answer pair based on the attribute value information.

7. The method according to claim 5, characterized in that The generating an extended question-answer pair based on the target question template, the attribute name, and the attribute value information includes: If the target question template is a numerical comparison template, and the numerical comparison template includes two fields to be compared, the two fields to be compared are replaced based on the attribute name and the attribute name in another key-value pair to obtain the question of the extended question-answer pair; and the answer to the extended question-answer pair is determined based on the comparison result of the attribute value information and the attribute value information in the other key-value pair; Alternatively, if the numerical comparison template includes a field to be compared and a reference numerical value, the field to be compared is replaced based on the attribute name to obtain the question of the extended question and answer pair; and the answer to the extended question and answer pair is determined based on the comparison result of the attribute value information and the reference numerical value; Alternatively, if the target question template is a value confirmation template, and the value confirmation template includes a field to be compared and a reference value, then the field to be compared in the value confirmation template is replaced based on the attribute name to obtain the question of the extended question and answer pair; based on the comparison result between the attribute value information and the reference value, the answer to the extended question and answer pair is determined; Alternatively, if the target question template is a maximum value comparison type template, and the maximum value comparison type template includes a reference value, then the question of the extended question and answer pair is determined based on the target question template; and the answer to the extended question and answer pair is determined based on the attribute value information in all key-value pairs and the reference value.

8. The method according to claim 5, characterized in that The method of obtaining a key-value pair based on the original non-tabular input data includes: inputting the original non-tabular input data and a second type of prompt word into a large model, wherein the second type of prompt word is used to guide the large model to provide a key-value pair according to the original non-tabular input data; generating a key-value pair corresponding to the original non-tabular input data based on the second type of prompt word and the original non-tabular input data by the large model, and outputting the key-value pair; The method further includes: inputting the first type of prompt words into the large model, generating an original question-answer pair corresponding to the original non-tabular input data based on the first type of prompt words and the original non-tabular input data by the large model, and outputting the original question-answer pair; The generating a target question-answer pair corresponding to the original input data based on the extended question-answer pair includes: verifying the original question-answer pair and the extended question-answer pair to obtain a target question-answer pair.

9. A question-answer pair acquisition device, characterized in that: The device comprises: an acquisition module, configured to acquire original input data; if the original input data includes original table input data, for each candidate cell in the original table input data, acquire the row title, column title and cell content corresponding to the candidate cell from the original table input data; A generating module, for generating a first question and a second question corresponding to the candidate cell based on the row header and the column header; wherein, in the first question, the row header is located in front of the column header, and in the second question, the column header is located in front of the row header; wherein, the first question is, how much is the row header + the column header +, and the second question is, how much is the column header + the row header +; A determination module, configured to determine a reference question corresponding to the candidate cell based on the semantic coherence of the first question and the semantic coherence of the second question; wherein, if the semantic coherence of the first question is greater than the semantic coherence of the second question, the reference question is the first question; and if the semantic coherence of the first question is less than the semantic coherence of the second question, the reference question is the second question; A processing module is used to generate an extended question-answer pair based on the reference question and the cell content; wherein the question of the extended question-answer pair is the reference question, and the answer to the extended question-answer pair is the cell content; and a target question-answer pair corresponding to the original input data is generated based on the extended question-answer pair.

10. An electronic device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the method described in any one of claims 1-8.

Images