Example selection method, training method, device and equipment of question identification model

Abstract

The application provides an example selection method, a problem identification model training method, device and equipment. The method comprises: performing problem intention identification on input text to obtain target problem text; determining a target problem data table corresponding to the target problem text; wherein the target problem data table comprises at least one example, and the example comprises a question text and a query statement; obtaining a first similarity between the target problem text and the question text of the example in the target problem data table, and obtaining a second similarity between a reference query statement and the question text of the example in the target problem data table; wherein the reference query statement is a query statement obtained by inputting the target problem text into a large language model; and obtaining a target example in the examples of the target problem data table based on the first similarity and the second similarity of the examples in the target problem data table. The method of the application can improve the efficiency of example selection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an example selection method, a training method for a problem identification model, an apparatus and device. Background Technology

[0002] The Text-to-SQL task transforms natural language questions in text into SQL queries using a Large Language Model (LLM). This lowers the barrier to entry for database use and greatly improves data processing efficiency. When processing Text-to-SQL tasks, appropriate examples are provided in the input prompts of the LLM, thereby improving the accuracy of the query statements output by the LLM.

[0003] Some example selection schemes for Text-to-SQL tasks involve manually picking examples from pre-labeled examples and applying them to all question texts. However, this approach results in low accuracy of the query statements output by the large language model and incurs high manual costs. Summary of the Invention

[0004] This application provides an example selection method, a problem identification model training method, apparatus, and device to solve the problems of high cost and low efficiency in example selection.

[0005] Firstly, this application provides an example selection method, including:

[0006] The input text is subjected to question intent recognition to obtain the target question text;

[0007] Determine the target question data table corresponding to the target question text; wherein, the target question data table includes at least one example, and the example includes the question text and the query statement;

[0008] Obtain the first similarity between the target question text and the example question text in the target question data table, and obtain the second similarity between the reference query statement and the example question text in the target question data table; wherein, the reference query statement is the query statement obtained by inputting the target question text into the large language model;

[0009] The target example is obtained from the examples in the target question data table based on the first similarity and the second similarity of the examples in the target question data table.

[0010] In one possible implementation, the number of input texts is multiple, and the step of performing question intent recognition on the input texts to obtain target question text includes:

[0011] Multiple texts are sequentially input into a pre-trained question recognition model to obtain the target question text for each text; among them...

[0012] The target question text of the first text indicates the question intent of the first text, and the target question text of the second text indicates the question intent of the second text, or indicates the question intent of the second text and the text input to the pre-trained question recognition model before the second text;

[0013] The first text is the first text input to the pre-trained question recognition model, and the second text is the text other than the second text among the plurality of texts.

[0014] In one possible implementation, after obtaining a target example based on a first similarity and a second similarity to examples in the target question data table, the method further includes:

[0015] Obtain prompt information based on the target example;

[0016] The prompt information and the target question text are input into the large language model to obtain the target query statement of the target question text.

[0017] In one possible implementation, obtaining the prompt information based on the target example includes:

[0018] Generate instructions; wherein, the instructions are used to instruct the large language model to generate a target query statement based on the target example and the target question data table in which the target example is located;

[0019] The instruction, the target example, and the target problem data table containing the target example are used as the prompt information.

[0020] In one possible implementation, the number of examples in the target problem data table is multiple; the step of obtaining a target example from the examples in the target problem data table based on a first similarity and a second similarity includes:

[0021] Assign a first weight to the first similarity and a second weight to the second similarity;

[0022] The similarity value of the corresponding example is determined based on the first similarity, second similarity, first weight, and second weight of each example;

[0023] The target example is determined from multiple examples based on the similarity scores of each example.

[0024] In one possible implementation, determining the target question data table corresponding to the target question text includes:

[0025] The target question text and multiple question data tables are input into a pre-trained pattern recognition model to obtain the target question data table; wherein,

[0026] The pre-trained pattern recognition model is obtained by inputting the training question text and the multiple question data tables into the initial pattern recognition model, and training it based on the training labels of the training question text.

[0027] Secondly, this application provides a training method for a problem identification model, including:

[0028] Obtain a first training set and a second training set; wherein, the first training set includes multiple first training texts, and there is a sequence relationship among the multiple first training texts, the first label of the first first training text in the sequence relationship represents the true question intent of the first first training text, and the first label of the non-first first training text in the sequence relationship represents the true question intent of the corresponding first training text and the first training text located before the corresponding first training text; the second training set includes multiple second training texts and a second label of each second training text, the second label representing the true question intent of the corresponding second training text;

[0029] The first training text in the first training set and the second training text in the second training set are sequentially input into the initial question recognition model. The initial question recognition model is trained based on the labels corresponding to each text in the first training text and the second training text to obtain the question recognition model. The question recognition model is used to recognize the question intent of the input text.

[0030] Thirdly, this application provides an example selection device, including: an intent recognition module, used to perform question intent recognition on the input text to obtain target question text;

[0031] The data table acquisition module is used to determine the target question data table corresponding to the target question text; wherein, the target question data table includes at least one example, and the example includes question text and query statement;

[0032] The similarity calculation module is used to obtain a first similarity between the target question text and the example question text in the target question data table, and to obtain a second similarity between a reference query statement and the example question text in the target question data table; wherein, the reference query statement is the query statement obtained by inputting the target question text into the large language model;

[0033] The target example acquisition module is used to acquire target examples from the examples in the target question data table based on the first similarity and the second similarity of the examples in the target question data table.

[0034] Fourthly, this application provides an electronic device, which includes: a processor and a memory; the memory is used to store instructions; the processor is used to execute the instructions in the memory, causing the electronic device to perform the methods described in the first and second aspects.

[0035] Fifthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the first and second aspects.

[0036] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in the first and second aspects.

[0037] The example selection method, question recognition model training method, apparatus, and equipment provided in this application determine the target text question corresponding to the input text by performing intent recognition on the input text. Based on the target text question, a target question data table is determined. Subsequently, examples of target question texts are selected from the examples in the target question data table. The similarity between the example question text and the target question text is calculated, and the similarity between the question query statement predicted by the large language model and the query statement of the example is calculated. Through these two similarities, the target question text target examples are determined in a low-cost, efficient, and accurate manner. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] Figure 1 An implementation scenario diagram illustrating an exemplary embodiment;

[0040] Figure 2 A flowchart illustrating an example selection method as an exemplary embodiment;

[0041] Figure 3 A flowchart illustrating a training method for an intent recognition model, as shown in an exemplary embodiment;

[0042] Figure 4 A structural diagram of an example selection device is shown as an exemplary embodiment;

[0043] Figure 5 A block diagram illustrating an electronic device as an exemplary embodiment.

[0044] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0046] It should be noted that the texts, data tables, examples, and other data involved in this application (including but not limited to data used for analysis, stored data, and displayed data) are all information and data that have been understood and authorized by the relevant users or have been fully authorized by all parties. Furthermore, the collection, use, processing, transmission, provision, disclosure, and application of the relevant data have all complied with relevant laws, regulations, and standards, taken necessary confidentiality measures, and have not violated public order and good morals, and have conformed to the principles of legality, legitimacy, and necessity.

[0047] First, let me explain the terms used in this application:

[0048] Large Language Model (LLM) User Profile: Composed of an artificial neural network with many parameters (usually billions or more weights), the large language model is generally a general model and is not targeted at a specific task scenario.

[0049] Text-to-SQL: Converts natural language used in communication into executable SQL query statements. SQL is a structured query language.

[0050] In-Context Learning (ICL): Language models learn tasks by using a set of examples or instructions organized in a demonstrative manner.

[0051] Large Language Models (LLMs) are commonly used in the field of natural language processing. LLMs can be pre-trained on massive amounts of text corpora and can perform many downstream tasks, such as text-to-SQL tasks. The purpose of text-to-SQL is to automatically transform natural language questions into SQL query statements. It can build a bridge between non-technical professionals and database systems. By converting the input text into query statements, data in the database can be queried through the query statements, which lowers the threshold for database use and greatly improves data processing efficiency.

[0052] Large language models (LLMs) possess excellent In-Context Learning (ICL) capabilities, meaning they can learn task patterns and output corresponding results solely from contextual examples. The key to developing ICL capabilities lies in providing appropriate and effective examples in the prompts input into the large model. Traditional ICL prompting involves manually selecting examples from labeled data and applying them to all test samples. This approach is both labor-intensive and limits the ICL performance of large models. Therefore, it is necessary to select suitable example data from labeled data for different test samples to help the large model achieve better performance when completing tasks.

[0053] It is important to consider that text-to-SQL tasks differ from general LLM natural language question answering tasks. The scenario is unique, requiring output examples when inputting into a large language model. When performing text-to-SQL conversion, the large language model will refer to the input examples to complete the task, achieving more accurate and efficient text-to-SQL conversion capabilities.

[0054] Some example selection methods only consider question similarity, which leads to low accuracy and high cost. In addition, there may be multiple database scenarios. When converting text to SQL, it is difficult to select appropriate examples in multiple database scenarios. This can easily lead to inconsistencies between the selected examples and the database to which the user's question belongs. This can cause the large model to understand the database structure and generate additional noise in the query statement, resulting in low accuracy of the large language model output. Furthermore, this method is not suitable for multi-round text to SQL scenarios.

[0055] Based on this, this embodiment proposes an example selection method, a training method for a problem identification model, an apparatus and equipment, aiming to solve the problems of low accuracy, high cost and low efficiency in example selection.

[0056] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0057] The training method and example selection method for the problem identification model provided in this application can be applied to... Figure 1 The implementation environment shown is as follows: Figure 1 As shown, the implementation environment may include a terminal and a server, and the terminal and the server are connected for communication.

[0058] Understandable. Figure 1 The number of terminals and servers shown is 1. In some embodiments, the number of terminals can be one or more, and in other embodiments, the number of servers can be one or more.

[0059] In this embodiment, the terminal can receive text, which may be text entered by the user on the terminal. The terminal sends the text to the server, and the server selects an example based on the text to obtain a target example. The target example can be returned to the terminal for display through the terminal's built-in visualization structure.

[0060] In some embodiments, after receiving text, the server performs question intent recognition on the input text to obtain target question text; determines the target question data table corresponding to the target question text; wherein the target question data table includes at least one example, and the example includes question text and query statement; obtains a first similarity between the target question text and the question text of the example in the target question data table, and obtains a second similarity between the reference query statement and the question text of the example in the target question data table; wherein the reference query statement is the query statement obtained by inputting the target question text into a large language model; and obtains a target example from the examples in the target question data table based on the first and second similarities of the examples in the target question data table.

[0061] In other embodiments, the terminal may receive a first training set and a second training set, and send the first training set and the second training set to the server, so that the server can train a question recognition model based on the first training set and the second training set, which can be used to identify the question intent of the text.

[0062] In some embodiments, the server obtains a first training set and a second training set; wherein, the first training set includes multiple first training texts, and there is a sequence relationship among the multiple first training texts, the first label of the first first training text in the sequence relationship represents the true question intent of the first first training text, and the first label of the non-first first training text in the sequence relationship represents the true question intent of the corresponding first training text and the first training text located before the corresponding first training text; the second training set includes multiple second training texts and a second label of each second training text, the second label representing the true question intent of the corresponding second training text; the first training texts in the first training set and the second training texts in the second training set are sequentially input into an initial question recognition model, so as to train the initial question recognition model based on the labels corresponding to each text in the first training text and the second training text, to obtain a question recognition model, the question recognition model is used to recognize the question intent of the input text.

[0063] In some embodiments, the terminal can be a wired terminal with a visual structure or a wireless terminal. In other embodiments, the terminal can be an electronic device with a visual structure, such as a mobile phone, computer, tablet, or vehicle-mounted device.

[0064] In some embodiments, the server may be a physical server, a server cluster, or a cloud server, etc., without specific limitations.

[0065] Understandably, the example selection device and the training device for the problem recognition model can be set up in... Figure 1 The server in the example, but as shown in this embodiment Figure 1 The implementation environment shown is merely exemplary. In other embodiments, the example selection method and the problem identification model training method can also be applied to other implementation environments. The example selection device and the problem identification model training device can also be set in other structures in other implementation environments, without specific limitations here.

[0066] Figure 2 This is a flowchart illustrating an exemplary selection method, applied to... Figure 1 Servers in, such as Figure 2 As shown, the method includes steps S210 to S270, which are described in detail below:

[0067] Step S210: Perform question intent recognition on the input text to obtain the target question text.

[0068] In this embodiment, problem intent recognition involves processing the input text to obtain a more direct and standard problem statement. For example, it involves judging the true inquiry intent of the current text by combining the context, and converting the text into a more direct target problem text through methods such as subject completion.

[0069] In some embodiments, the input text can be multi-round, that is, the number of input texts is multiple. These multiple texts are sequentially subjected to question intent recognition to obtain the target question text corresponding to each text. It can be understood that the number of texts is multiple, which means that each text in the multiple texts is subjected to corresponding example selection and enters the large language model to obtain the target query statement. The multiple texts are the example selection in multiple rounds.

[0070] In some embodiments, question intent recognition is achieved through a pre-trained question recognition model. By inputting text into the question recognition model, the target question text is obtained.

[0071] Of course, when there are multiple input texts, these multiple texts are sequentially fed into a pre-trained question recognition model to obtain the target question text corresponding to each text.

[0072] In some embodiments, the question intents corresponding to multiple texts may be related. For example, the text currently entering the question recognition model is proposed based on the text previously input to the question recognition model. Thus, when the question recognition model performs question intent recognition on the current text, it will also analyze the relationship between the text previously input to the question recognition model and the current text, thereby outputting the question intent of the current text, that is, the target question text.

[0073] If the first text input into the pre-trained question recognition model from among multiple texts is taken as the first text, and the text other than the second text from among the multiple texts is taken as the second text, then the target question text of the first text indicates the question intent of the first text, the target question text of the second text indicates the question intent of the second text, or indicates the question intent of the second text and the text input into the pre-trained question recognition model before the second text.

[0074] In other words, apart from the first text input to the pre-trained question recognition model, subsequent texts may be related to the text previously input to the pre-trained question recognition model. The output target question text indicates the question intent of the text and the text input to the pre-trained question recognition model before it. However, subsequent texts may not be related to the text previously input to the pre-trained question recognition model. The output target question text indicates the question intent of the text.

[0075] For example, in some embodiments, if the first input text to the pre-trained question recognition model is "Which city has the largest population?", then the target question text indicates the question intent of "Which city has the largest population?". If the second input text to the pre-trained question recognition model is "Which province is it located in? Tell me the name and area of ​​the province.", and there is a connection between the second input text and the first input text, then the target question text indicates the question intent of "Which province does the city with the largest population belong to? Tell me the name and area of ​​the province", which also indicates the question intent of the first input text to the pre-trained question recognition model.

[0076] Step S230: Determine the target question data table corresponding to the target question text; wherein, the target question data table includes at least one example, and the example includes the question text and the query statement.

[0077] In this embodiment, after obtaining the target question text, the target question data table corresponding to the target question text can be determined. That is, pattern recognition is performed on the target question text, and the target question data table corresponding to the target question text is determined among multiple question data tables. This reduces the interference of other question data tables on the example selection of the target question text, and subsequent example selection or conversion to target query statements only processes the data in the target question data table, thereby improving efficiency.

[0078] Understandably, there may be one or more target question data tables. When there is only one, the target question data table is the data table for querying the question corresponding to the text to obtain the query results. If there are multiple, further example selection is required. The target question data table where the target example is located is the data table for querying the question corresponding to the text to obtain the query results.

[0079] In some embodiments, the target question data table of the text is predicted by a pattern recognition model. In some embodiments, the target question text and multiple question data tables are input into a pre-trained pattern recognition model to obtain the target question data table. The pre-trained pattern recognition model is obtained by inputting the training question text and multiple question data tables into an initial pattern recognition model to train based on the training labels of the training question text.

[0080] The training question text can be the target question text obtained by performing question intent recognition on the training text. The training label corresponding to the training question text is at least one question data table from multiple question data tables. That is, the training question text and multiple question data tables are input into the initial pattern recognition model. The initial pattern recognition model outputs the predicted question data table, and the initial pattern recognition model is trained based on the predicted question data table and the question data table corresponding to the training label.

[0081] In this embodiment, each problem data table includes at least one example. The examples corresponding to different problem data tables may be the same or different. It is understood that the examples corresponding to a problem data table are related to the attributes and data in that problem data table.

[0082] The example includes a question text and a query statement. An example includes a question text and a query statement. That is, the question text in an example corresponds to a query statement, and the query statement is related to the data in the corresponding question data table, such as the row and column attributes in the question data table. The structure in the query statement is associated with the row and column attributes, so that the data in the question data table can be queried through the query statement.

[0083] The target question text, "Which city has the largest population?", yields a single target question data table, named "City". This table includes attributes such as "City ID" (the primary key), "Name" (showing the name of each city), "Area" (showing the area of ​​each city), and "Population" (showing the population of each city). This target question data table can also have relationships with other question data tables, allowing access to data in other tables. Correspondingly, the query statements in this target question data table, such as conditional statements and specified columns, are generated based on this table.

[0084] Step S250: Obtain the first similarity between the target question text and the example question text in the target question data table, and obtain the second similarity between the reference query statement and the example question text in the target question data table.

[0085] In this embodiment, after determining the target question data table corresponding to the target question text, the target example corresponding to the target question text can be determined from the examples in the target question data table. The target example needs to clearly demonstrate the mapping relationship between the target question text and the target query statement that the target question text needs to be converted into. In this way, the large language model can learn the mapping relationship between the question text and the query statement in the target example and accurately convert the target question text into the target query statement.

[0086] In this embodiment, the first similarity between the target question text and the example question text in the target question data table is considered, and the second similarity between the reference query statement and the example question text in the target question data table is considered. The target example corresponding to the target question text is determined by the two similarities.

[0087] The reference query statement is the query statement obtained by inputting the target question text into the large language model. At this time, the large language model receives the target question text and outputs the reference query statement. Of course, the large language model can also accept the target question text and prompts instructing it to convert the target question text into a query statement, and output the reference query statement accordingly. Understandably, the large language model does not accept examples of the target question text at this time. Therefore, the obtained reference query statement is not the final result of converting the target question text into a query statement, but is only used to filter query statements with high similarity among the examples in the target question data table.

[0088] Step S270: Obtain the target example from the examples in the target problem data table based on the first and second similarity of the examples in the target problem data table.

[0089] In this embodiment, the number of examples in the target problem data table is at least one. A first weight is assigned to the first similarity, and a second weight is assigned to the second similarity. The similarity value of the corresponding example is determined based on the first similarity, second similarity, first weight, and second weight of each example. The target example is determined from multiple examples based on the magnitude of the similarity value of each example.

[0090] In some embodiments, the similarity values ​​of the examples can be calculated in the following manner:

[0091] S i =k1P(q * q i )+k2P(sql * SQL i )

[0092] Among them, S i Let q be the similarity value of the i-th example, k1 be the first weight, k2 be the second weight, and q be the second weight. * For the target question text, q i Let P(q) be the question text for the i-th example. * q i Let P(sql) be the first similarity score of the i-th example, which can be the likelihood function score. * SQL i ) represents the second similarity of the i-th example, sql * For reference, SQL i Let i be the query statement for the i-th example.

[0093] In this embodiment, the example with the highest similarity value is selected as the target example.

[0094] This embodiment proposes an example selection method. Intent recognition is performed on the input text to determine the target text question. Based on this, a target question data table is determined. Subsequently, examples of the target question text are selected from the examples in this target question data table, reducing the amount of data processed in subsequent example selection. A similarity calculation is then performed between the example text and the examples in the target question data table. This calculation includes not only the similarity of the question text but also the similarity between the question query statement predicted by the large language model and the query statement of the example text. By using these two similarity metrics, the target examples of the target question text are determined efficiently and accurately, and the process is low-cost.

[0095] Furthermore, it is proposed that when there are multiple text data, the intent recognition results obtained from the previous text will be referenced when performing intent recognition on the text, thereby improving the accuracy of intent recognition and improving the accuracy of subsequent example selection.

[0096] In some embodiments, after obtaining a target example, the target question text can be transformed into a query statement using that target example. This transformation process may include:

[0097] S10: Obtain prompt information based on the target example.

[0098] S11: Input the prompt information and the target question text into the large language model to obtain the target query statement of the target question text.

[0099] In this embodiment, the prompt information can instruct the large language model on the operation required for the input target question text. The prompt information includes instructions, target examples, and a target question data table.

[0100] In some embodiments, an instruction is first generated; wherein the instruction is used to instruct the large language model to generate a target query statement based on the target example and the target question data table in which the target example is located; the instruction, the target example, and the target question data table in which the target example is located are used as prompt information.

[0101] In one embodiment, the target question text is "Which city has the largest population?", the target question data table is named "City", and the resulting target example is "Question text: Which city has the smallest population? Query statement: SELECT field name FROM city ORDER BY population ASC LIMIT 1". In this query statement, SELECT is used to query field names, FROM is used to identify the name of the target question data table, ORDER BY indicates the sorting of the corresponding fields, ORDER BY population means sorting the data of the population attribute in the target question data corresponding to the city, ASC LIMIT means that the query results are sorted in ascending order, and 1 means taking the top 1 record.

[0102] The corresponding instruction could be "Give the SQL statement for the target question text based on the structure and target example in the target question data table named 'City'". Inputting the prompt and the target question text into the large language model will generate the target query statement.

[0103] It is understandable that the target problem data table in the prompt message is the data table where the target example is located. That is, when there are multiple target problem data tables obtained in step S230, the generated prompt message only includes the target problem data table where the last obtained target example is located.

[0104] Figure 3 This is a flowchart illustrating an exemplary embodiment of a training method for a problem identification model, which can be applied to... Figure 1 In the servers shown, such as Figure 3 As shown, the method includes steps S310 to S330, which are explained in detail below:

[0105] Step S310: Obtain a first training set and a second training set; wherein the first training set includes multiple first training texts and the second training set includes multiple second training texts.

[0106] In this embodiment, the training of the intent recognition model may include a first training set and a second training set. The training texts in the first training set are related to each other, that is, the questions in the first training text are related to the questions in the second training text, while the questions in the training texts in the second training set are not related to each other.

[0107] For each training text in the first training set, there is a sequence relationship between multiple first training texts. This sequence relationship represents the training order of each training text in the first training set as input to the initial intent recognition model. The first label of the first first training text in the sequence relationship represents the true question intent of the first first training text. The first label of a non-first first training text in the sequence relationship represents the true question intent of the corresponding first training text and the first training text located before the corresponding first training text.

[0108] In this embodiment, the first label of each first training text in the first training set, except for the first first training text, represents the true question intent of the first training text itself and the first training text preceding it. It can be understood that since the true question intent of each first training text in the first training set, except for the first first training text, includes the true question intent of the preceding first training text, even if a certain first training text is actually related to the questions of all the first training texts preceding it, it is only necessary to refer to the true question intent of the first training text preceding it. For example, there are 3 training texts in the first training set. For the 3rd first training text, it is related to the first 2 first training texts, but its corresponding first label represents the true question intent of the corresponding first training text and the 2nd first training text, because the first label of the 2nd first training text represents the true question intent of the 2nd first training text and the 1st first training text.

[0109] For the second training set, which includes multiple second training texts and a second label for each second training text, the second label represents the true question intent of the corresponding second training text.

[0110] Step S330: Input the first training text in the first training set and the second training text in the second training set into the initial question recognition model in sequence, so as to train the initial question recognition model based on the labels corresponding to each text in the first training text and the second training text, and obtain the question recognition model.

[0111] In this embodiment, the first training text in the first training set and the second training text in the second training set are sequentially input into the initial question recognition model. When the first training text in the first training set is input into the initial question recognition model, the initial question recognition model inputs a first prediction label for each first training text. The initial question recognition model is trained by the first prediction label and the corresponding first label. The first training set is used to enable the initial question recognition model to learn how to learn the true question intent of the text when there is a correlation between the questions in multiple texts.

[0112] When the second training text in the second training set is input into the initial question recognition model, the initial question recognition model inputs the second predicted label for each second training text, and trains the initial question recognition model using the second predicted label and the corresponding second label. The second training set is used to enable the initial question recognition model to learn how to learn the true question intent of the text when there is no correlation between the questions in multiple texts.

[0113] The question recognition model is used to identify the question intent of the input text, as in step S210.

[0114] Figure 4 This is an exemplary embodiment illustrating a structural diagram of an example selection device. The example selection device 400 includes:

[0115] The intent recognition module 410 is used to recognize the question intent of the input text and obtain the target question text;

[0116] The data table acquisition module 430 is used to determine the target question data table corresponding to the target question text; wherein, the target question data table includes at least one example, and the example includes the question text and the query statement;

[0117] The similarity calculation module 450 is used to obtain the first similarity between the target question text and the example question text in the target question data table, and to obtain the second similarity between the reference query statement and the example question text in the target question data table; wherein, the reference query statement is the query statement obtained by inputting the target question text into the large language model;

[0118] The target example acquisition module 470 is used to acquire target examples from the examples in the target question data table based on the first similarity and the second similarity of the examples in the target question data table.

[0119] In one possible implementation, the number of input texts is multiple, and the intent recognition module includes:

[0120] The intent recognition unit is used to sequentially input multiple texts into a pre-trained question recognition model, thereby obtaining the target question text corresponding to each text; whereby...

[0121] The target question text of the first text indicates the question intent of the first text, and the target question text of the second text indicates the question intent of the second text, or indicates the question intent of the second text and the text input to the pre-trained question recognition model before the second text;

[0122] The first text is the first text input into the pre-trained question recognition model, and the second text is the text other than the second text among multiple texts.

[0123] In one possible implementation, the example selection device further includes:

[0124] The prompt information acquisition module is used to obtain prompt information based on the target example;

[0125] The query statement acquisition module is used to input the prompt information and the target question text into the large language model to obtain the target query statement of the target question text.

[0126] In one possible implementation, the prompt information acquisition module includes:

[0127] The instruction generation unit is used to generate instructions; wherein, the instructions are used to instruct the large language model to generate a target query statement based on the target example and the target question data table in which the target example is located;

[0128] The prompt information acquisition unit is used to obtain the instruction, the target example, and the target problem data table where the target example is located as prompt information.

[0129] In one possible implementation, the target problem data table contains multiple examples; the target example acquisition module includes:

[0130] A weight allocation unit is used to assign a first weight to the first similarity and a second weight to the second similarity;

[0131] The similarity value calculation unit is used to determine the similarity value of the corresponding example based on the first similarity, second similarity, first weight, and second weight of each example;

[0132] The target example acquisition unit is used to determine the target example from multiple examples based on the similarity values ​​of each example.

[0133] In one possible implementation, the data table retrieval module includes:

[0134] The data table acquisition unit is used to input the target question text and multiple question data tables into a pre-trained pattern recognition model to obtain the target question data table; wherein,

[0135] The pre-trained pattern recognition model is obtained by inputting the training question text and multiple question data tables into the initial pattern recognition model, and training it based on the training labels of the training question text.

[0136] The example selection device provided in this embodiment can be used to execute the example selection method described above. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0137] It is understandable that a training device for a problem identification model may also be shown, comprising:

[0138] The training data acquisition module is used to acquire a first training set and a second training set. The first training set includes multiple first training texts, which have a sequential relationship. The first label of the first first training text in the sequential relationship represents the true question intent of the first first training text. The first label of the non-first first training text in the sequential relationship represents the true question intent of the corresponding first training text and the first training text preceding the corresponding first training text. The second training set includes multiple second training texts and a second label of each second training text. The second label represents the true question intent of the corresponding second training text.

[0139] The training module is used to sequentially input the first training text from the first training set and the second training text from the second training set into the initial question recognition model, so as to train the initial question recognition model based on the labels corresponding to each text in the first and second training texts, and obtain the question recognition model. The question recognition model is used to recognize the question intent of the input text.

[0140] The training device for the problem identification model provided in this embodiment can be used to execute the training method of the above-mentioned problem identification model. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0141] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Please refer to [link / reference]. Figure 5 The electronic device 500 may include a processor 51 and a memory 52, wherein the processor 51 and the memory 52 can communicate; for example, the processor 51 and the memory 52 communicate via a communication bus 53, the memory 52 is used to store instructions, and the processor 51 is used to call the instructions in the memory to execute the training method or example selection method of the problem identification model shown in any of the above method embodiments.

[0142] The aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0143] This application provides a computer-readable storage medium storing computer-executable instructions; when executed by a processor, the computer-executable instructions are used to implement a training method or an example selection method for a problem identification model as described in any of the above embodiments.

[0144] This application provides a computer program product, which includes a computer program that, when executed by a processor, implements the training method or example selection method of the above-mentioned problem identification model.

[0145] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0146] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An example selection method, characterized in that, include: The input text is subjected to question intent recognition to obtain the target question text; Determine the target question data table corresponding to the target question text; wherein, the target question data table includes at least one example, and the example includes the question text and the query statement; Obtain the first similarity between the target question text and the example question text in the target question data table, and obtain the second similarity between the reference query statement and the example question text in the target question data table; wherein, the reference query statement is the query statement obtained by inputting the target question text into the large language model; The target example is obtained from the examples in the target question data table based on the first similarity and the second similarity of the examples in the target question data table.

2. The method according to claim 1, characterized in that, The number of input texts is multiple, and the step of performing question intent recognition on the input texts to obtain the target question text includes: Multiple texts are sequentially input into a pre-trained question recognition model to obtain the target question text for each text; among them... The target question text of the first text indicates the question intent of the first text, and the target question text of the second text indicates the question intent of the second text, or indicates the question intent of the second text and the text input to the pre-trained question recognition model before the second text; The first text is the first text input to the pre-trained question recognition model, and the second text is the text other than the second text among the plurality of texts.

3. The method according to claim 1, characterized in that, After obtaining the target example based on the first and second similarities of the examples in the target question data table, the method further includes: Obtain prompt information based on the target example; The prompt information and the target question text are input into the large language model to obtain the target query statement of the target question text.

4. The method according to claim 3, characterized in that, The step of obtaining the prompt information based on the target example includes: Generate instructions; wherein, the instructions are used to instruct the large language model to generate a target query statement based on the target example and the target question data table in which the target example is located; The instruction, the target example, and the target problem data table containing the target example are used as the prompt information.

5. The method according to claim 1, characterized in that, The number of examples in the target question data table is multiple; the process of obtaining target examples from the examples in the target question data table based on the first and second similarities of the examples in the target question data table includes: Assign a first weight to the first similarity and a second weight to the second similarity; The similarity value of the corresponding example is determined based on the first similarity, second similarity, first weight, and second weight of each example; The target example is determined from multiple examples based on the similarity scores of each example.

6. The method according to claim 1, characterized in that, The step of determining the target question data table corresponding to the target question text includes: The target question text and multiple question data tables are input into a pre-trained pattern recognition model to obtain the target question data table; wherein, The pre-trained pattern recognition model is obtained by inputting the training question text and the multiple question data tables into the initial pattern recognition model, and training it based on the training labels of the training question text.

7. A training method for a problem identification model, characterized in that, include: Obtain a first training set and a second training set; wherein, the first training set includes multiple first training texts, and there is a sequence relationship among the multiple first training texts, the first label of the first first training text in the sequence relationship represents the true question intent of the first first training text, and the first label of the non-first first training text in the sequence relationship represents the true question intent of the corresponding first training text and the first training text located before the corresponding first training text; the second training set includes multiple second training texts and a second label of each second training text, the second label representing the true question intent of the corresponding second training text; The first training text in the first training set and the second training text in the second training set are sequentially input into the initial question recognition model. The initial question recognition model is trained based on the labels corresponding to each text in the first training text and the second training text to obtain the question recognition model. The question recognition model is used to recognize the question intent of the input text.

8. An example selection device, characterized in that, include: The intent recognition module is used to recognize the question intent of the input text and obtain the target question text; The data table acquisition module is used to determine the target question data table corresponding to the target question text; wherein, the target question data table includes at least one example, and the example includes question text and query statement; The similarity calculation module is used to obtain a first similarity between the target question text and the example question text in the target question data table, and to obtain a second similarity between a reference query statement and the example question text in the target question data table; wherein, the reference query statement is the query statement obtained by inputting the target question text into the large language model; The target example acquisition module is used to acquire target examples from the examples in the target question data table based on the first similarity and the second similarity of the examples in the target question data table.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory; the memory is used to store instructions; the processor is used to execute the instructions in the memory, causing the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images