Intelligent question and answer method, related device, equipment and storage medium

Abstract

The invention discloses an intelligent question and answer method, a related device, equipment and a storage medium, and the intelligent question and answer method comprises the steps: carrying out the retrieval based on a to-be-replied first statement, and obtaining a plurality of retrieval results; inputting the first statement and the plurality of retrieval results to the intelligent dialogue model to obtain an output statement of the intelligent dialogue model as a second statement for replying the first statement; wherein the intelligent dialogue model is trained based on sample data, and the sample data comprises a first sample statement, a second sample statement correctly replying the first sample statement and a plurality of sample retrieval results of the first sample statement; the plurality of sample retrieval results at least comprise a positive example retrieval result based on the first sample statement retrieval and a negative example retrieval result based on the sample negative example statement retrieval, and replacing the entity in the first sample statement to obtain the sample negative example statement. According to the scheme, the accuracy of model answering can be improved in a retrieval enhancement scene.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to an intelligent question-answering method and related devices, equipment and storage media. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent dialogue models represented by large language models have been widely used in many scenarios and industries.

[0003] In order to solve the problems of outdated information and model hallucination in intelligent dialogue models, model generation solutions based on retrieval enhancement have been widely used. Although retrieval results can provide reference information for model generation, retrieval results as external knowledge may also interfere with model generation. In particular, when the retrieval results are irrelevant to the input question, it will cause the model to have incorrect references, which will affect the accuracy of the model's output answers. In view of this, how to improve the accuracy of model answers in retrieval enhancement scenarios has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by this application is to provide an intelligent question-answering method and related devices, equipment and storage media, which can improve the accuracy of model answers in retrieval enhancement scenarios.

[0005] In order to solve the above technical problems, the first aspect of the present application provides an intelligent question-answering method, including: performing a search based on a first sentence to be replied to, and obtaining a number of search results; inputting the first sentence and the number of search results into an intelligent dialogue model, and obtaining an output sentence of the intelligent dialogue model as a second sentence to reply to the first sentence; wherein the intelligent dialogue model is trained based on sample data, and the sample data includes: a first sample sentence, a second sample sentence that correctly replies to the first sample sentence, and a number of sample retrieval results of the first sample sentence, and the number of sample retrieval results at least include a positive retrieval result based on the retrieval of the first sample sentence and a negative retrieval result based on the retrieval of the sample negative sentence, and the sample negative sentence is obtained by replacing the entity in the first sample sentence.

[0006] In order to solve the above-mentioned technical problems, the second aspect of the present application provides an intelligent question-answering device, including: a question retrieval module and a question answering module, the question retrieval module is used to perform retrieval based on a first sentence to be replied to, and obtain a number of retrieval results; the question answering module is used to input the first sentence and the number of retrieval results into an intelligent dialogue model, and obtain an output sentence of the intelligent dialogue model as a second sentence to reply to the first sentence; wherein the intelligent dialogue model is trained based on sample data, and the sample data includes: a first sample sentence, a second sample sentence that correctly replies to the first sample sentence, and a number of sample retrieval results of the first sample sentence, and the number of sample retrieval results at least include a positive example retrieval result based on the retrieval of the first sample sentence and a negative example retrieval result based on the retrieval of the sample negative example sentence, and the sample negative example sentence is obtained by replacing the entity in the first sample sentence.

[0007] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, which at least includes a memory and a processor coupled to each other, wherein the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the intelligent question and answer method in the above first aspect.

[0008] In order to solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the intelligent question-answering method of the first aspect.

[0009] In the above scheme, a search is performed based on the first sentence to be replied to, and several search results are obtained. The first sentence and the several search results are then input into the intelligent dialogue model to obtain the output sentence of the intelligent dialogue model as the second sentence to reply to the first sentence. The intelligent dialogue model is trained based on sample data. The sample data includes: a first sample sentence, a second sample sentence that correctly answers the first sample sentence, and several sample search results of the first sample sentence. The several sample search results at least include positive search results based on the first sample sentence search and negative search results based on the sample negative sentence search. The sample negative sentence is obtained by replacing the entity in the first sample sentence. Therefore, by replacing the entity in the first sample sentence as the sample negative sentence, a negative search result that is extremely interfering with the intelligent dialogue model when answering the first sample sentence can be obtained based on the sample negative sentence. Therefore, during the training process, the intelligent dialogue model can use the second sample sentence that correctly answers the first sample sentence as the output target, learn to accurately distinguish between positive search results that are helpful in answering the input question and negative search results that are not helpful in answering the input question, and alleviate the model's incorrect reference problem as much as possible, thereby improving the accuracy of the model's answer in the retrieval enhancement scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of an embodiment of the intelligent question-answering method of the present application; Figure 2a is a schematic diagram of a process of obtaining candidate search results in one embodiment; Figure 2b is a schematic diagram of a process of training an embodiment of an intelligent dialogue model; Figure 3 It is a schematic diagram of the framework of an embodiment of the intelligent question-answering device of the present application; Figure 4 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application; Figure 5 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0011] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0012] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0013] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the fragment " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0014] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the intelligent question-answering method of the present application. Specifically, it may include the following steps: Step S11: Perform a search based on the first sentence to be replied to, and obtain a number of search results.

[0015] In an implementation scenario, the specific content of the first sentence may be different depending on the application scenario and the purpose of use, and the specific content of the first sentence is not limited. For example, taking the question-and-answer scenario of a sports event as an example, the first sentence may include but is not limited to: "How many gold medals did the A country team win at the Xth Games" etc.; or, taking the question-and-answer scenario of travel as an example, the first sentence may include but is not limited to: "Is Province B suitable for the elderly to go in winter?" etc. Of course, the above examples are only a few possible examples of the first sentence in actual application, and the specific content of the first sentence is not limited here.

[0016] In one implementation scenario, as a possible example, the first sentence can be input in text; or, as another possible example, the first sentence can be input in voice. The above examples are only several possible input methods of the first sentence, and the input method of the first sentence is not limited here.

[0017] In an implementation scenario, a search may be performed in a database based on the first sentence to obtain several search results. For example, the database may contain several document data, each of which may be pre-featured to obtain its document feature vector, and when searching based on the first sentence, the first sentence may also be feature extracted to obtain its sentence feature vector. On this basis, the similarity between the sentence feature vector and each document feature vector may be obtained, and then the similarities may be sorted in descending order, and the document data before the preset order may be selected as the search results.

[0018] In another implementation scenario, different from the above retrieval method, a search engine can be directly searched based on the first sentence to obtain several search results. It should be noted that the search engine can be any search engine, and the search engine used is not specifically limited here.

[0019] In another implementation scenario, different from the above two search methods, the above two search methods can also be combined to obtain a number of search results. That is, on the one hand, based on the first sentence, a search can be performed in the database to obtain a search result (e.g., M search results), and on the other hand, based on the first sentence, a search can be performed in the search engine to obtain a search result (e.g., N search results), so that the search results obtained in the two aspects can be combined to obtain a number of search results (e.g., M+N search results).

[0020] Of course, the above examples are only several possible ways of searching based on the first sentence, and the specific way of searching based on the first sentence is not limited here.

[0021] Step S12: input a first sentence and a number of search results into the intelligent dialogue model, and obtain an output sentence of the intelligent dialogue model as a second sentence in reply to the first sentence.

[0022] In an implementation scenario, the intelligent dialogue model may include but is not limited to open source large models such as Bloom, LLAMA, etc.; or, the intelligent dialogue model may include but is not limited to fine-tuning the parameters of the open source large model based on a specific corpus; or, the intelligent dialogue model may also include but is not limited to a custom large model. The above examples are only a few possible examples of intelligent dialogue models, and the specific selection of intelligent dialogue models is not limited here.

[0023] In an implementation scenario, taking the intelligent dialogue model using a large language model as an example, a prompt instruction can be constructed based on the first sentence and several search results, and then the prompt instruction is input into the intelligent dialogue model, so that the output of the intelligent dialogue model and the second sentence as a reply to the first sentence can be obtained. For example, taking the first sentence "How many gold medals did the A country team win at the Xth Games" as an example, the search results may include but are not limited to: "The A country team ranked first in the medal table at the Xth Games, leading all countries, including XX gold medals, XX silver medals, XX bronze medals, and a total of XX medals, leading all countries", "Team members of the A country won the first gold medal at the Xth Games", "The A country team won XX gold medals at the Yth Games", etc. On this basis, prompt instructions can be constructed: Please answer the question "How many gold medals did the A country team win in the Xth Games" based on the following search results: Search result 1: Country A's team ranked first in the medal table at the Xth Games, leading all countries. Among them, there were XX gold medals, XX silver medals, and XX bronze medals, with a total of XX medals, leading all countries; Search result 2: The team members of country A won the first gold medal in the Xth Games; Search result 3: The team of country A won XX gold medals at the Yth Games.

[0024] It should be noted that the above example is only a possible example of prompting instructions when the intelligent dialogue model is a large language model, and other possible situations are not given one by one. Still taking the above example as an example, the output sentence of the intelligent dialogue model can be "The A country team won XX gold medals in the Xth Games", which can be used as the second sentence to reply to the first sentence "How many gold medals did the A country team win in the Xth Games".

[0025] In the disclosed embodiment, the intelligent dialogue model can be trained based on sample data, the sample data includes: a first sample sentence, a second sample sentence that correctly responds to the first sample sentence, and several sample retrieval results of the first sample sentence, the several sample retrieval results at least include positive retrieval results based on the first sample sentence retrieval and negative retrieval results based on the sample negative sentence retrieval, and the sample negative sentence is obtained by replacing the entity in the first sample sentence.

[0026] In an implementation scenario, the specific meaning of the first sample sentence can refer to the aforementioned description of the first sentence, which will not be repeated here. In the process of obtaining a sample negative sentence based on the first sample sentence, each replacement can be only for one entity in the first sample sentence (such as a person's name, a place name, an institution name, a time, etc.). As a possible example, each entity in the first sample sentence can be selected as the target entity, and then based on the first sample sentence and the target entity, a first prompt instruction is constructed, and the first prompt instruction is used to instruct the large language model to replace the target entity in the first sample sentence. On this basis, the first prompt instruction can be input into the large language model to obtain the output sentence of the large language model as a sample negative sentence. Taking the first sample sentence "Is Province B suitable for the elderly in winter?" as an example, the entities may include "Province B", "Winter", "Elderly", etc., which can be selected as target entities respectively. When "Province B" is selected as the target entity, the first prompt instruction "Please replace the target entity 'Province B' in 'Is Province B suitable for the elderly to visit in winter' with other words with the same part of speech but completely different meanings" can be constructed in combination with the first sample sentence, and input into the large language model to obtain the output sentence of the large language model as the sample negative sentence "Is Province C suitable for the elderly to visit in winter"; or, when "winter" is selected as the target entity, the first prompt instruction "Please replace the target entity 'winter' in 'Is Province B suitable for the elderly to visit in winter' with other words with the same part of speech but completely different meanings" can be constructed in combination with the first sample sentence, and input into the large language model to obtain the output sentence of the large language model as the sample negative sentence "Is Province B suitable for the elderly to visit in summer"; or, when "elderly people" is selected as the target entity, the first prompt instruction "Please replace the target entity 'elderly people' in 'Is Province B suitable for the elderly to visit in winter' with other words with the same part of speech but completely different meanings" can be constructed in combination with the first sample sentence, and input into the large language model to obtain the output sentence of the large language model as the sample negative sentence "Is Province B suitable for young people to visit in winter". That is to say, in the embodiment of the present disclosure, the sample negative sentence and the first sample sentence can be similar in expression but different in meaning. It should be noted that the above examples are only several possible examples of sample negative sentences in actual application, and the specific content of the sample negative sentences is not limited here. In the above method, each entity in the first sample sentence is selected as the target entity, and then based on the first sample sentence and the target entity, a first prompt instruction is constructed, and the first prompt instruction is used to instruct the large language model to replace the target entity in the first sample sentence, and the first prompt instruction is input to the large language model to obtain the output sentence of the large language model as the sample negative sentence. The sample negative sentence can be constructed through the general understanding ability of the large language model, which helps to improve the quality of the sample negative sentence.

[0027] In one implementation scenario, after obtaining the sample negative sentence, a search can be performed based on the sample negative sentence, so that several first search results of the sample negative sentence can be obtained, and each of them can be used as the first negative result, and then at least based on the first negative result, several candidate search results can be obtained (for example, the first negative result can be directly selected as the candidate search result), and then based on the correlation between the first sample sentence and each candidate search result, the candidate search result can be selected as the negative search result. It should be noted that the candidate search result selected as the negative search result is required to have a low correlation with the first sample sentence, such as being sorted in order from low to high in correlation, and selecting the candidate search results before the preset order (for example, the first three, etc.) as the negative search result. In the above manner, by filtering the negative search results through correlation, the model can be further provided with search results that are irrelevant or low-correlated with the first sample sentence as much as possible, which is helpful for the intelligent dialogue model to enhance negative example learning.

[0028] In a specific implementation scenario, as a possible example, in addition to the aforementioned implementation method of obtaining candidate search results based on the first negative example result, the second search result that is located after the first rank (e.g., after the 20th place, or after the 50th place) when searching based on the first sample sentence can also be selected as the second negative example result. On this basis, the first negative example result and the second negative example result can be combined to obtain several candidate search results. In other words, the later search results among the various search results retrieved by the first sample sentence, as well as the various candidate searches retrieved based on the sample negative sentence, can all be used as candidate search results. Please refer to Figure 2a , Figure 2a FIG. 1 is a schematic diagram of a process for obtaining candidate search results according to an embodiment of the present invention. Figure 2a As shown, taking the first sample sentence "Is Province B suitable for elderly people to go in winter?" as an example, each first search result of its sample negative sentence "Is Province C suitable for elderly people to go in winter?" can be selected as the first negative result, and the second search result located after the first rank (e.g., after the 30th rank) when searching based on the first sample sentence "Is Province B suitable for elderly people to go in winter?" can be selected as the second negative result, so as to obtain several candidate search results (e.g., Figure 2a Of course, Figure 2a The search results shown are only a few possible examples in actual application, and the specific content of the search results is not limited here. In the above method, the second search result located after the first order when searching based on the first sample sentence is selected as the second negative example result, and the candidate search result is obtained by combining the first negative example result, which can further expand the selection range of the negative example search result.

[0029] In a specific implementation scenario, as a possible example, in order to facilitate the measurement of relevance to filter out negative retrieval results, a second prompt instruction can be constructed based on the first sample sentence and several candidate retrieval results, and the second prompt instruction is used to instruct the large language model to filter out the candidate retrieval results related to the first sample sentence, and then input the second prompt instruction to the large language model to obtain the candidate retrieval results output by the large language model as negative retrieval results. Please continue to refer to Figure 2a , still taking the first sample sentence "Is Province B suitable for elderly people to visit in winter?" as an example, several candidate search results can be obtained: "The average temperature in Province C is above 20 degrees in winter...", "Province C is the peak season for northern tourists to travel south in winter", "According to statistics, the number of elderly people traveling in Province B has reached a new high...", "Province B is covered in snow in winter...", then the following second prompt instruction can be constructed: Please evaluate the relevance between the search results and the question "Is Province B suitable for elderly people to visit in winter?" and filter out The search results related to answering the above questions, output the sequence numbers of the remaining search results: Search result 1: The average winter temperature in Province C is above 20 degrees...; Search result 2: Province C is the peak season for northern tourists to travel south in winter; Search result 3: According to statistics, the number of elderly people traveling in Province B has reached a new high...; Search result 4: Province B is covered in snow in winter…

[0030] Of course, the above example is only a possible example of the second prompt instruction in the actual application process, and the specific content of the second prompt instruction is not limited here. In addition, as a possible example, after the above second prompt instruction is input into the large language model, the output result can be obtained: retrieval result 1, retrieval result 2, retrieval result 3, and these three retrieval results can be retained as negative retrieval results. In the above method, based on the first sample sentence and several candidate retrieval results, the second prompt instruction is constructed, and the second prompt instruction is used to instruct the large language model to filter out the candidate retrieval results related to the first sample sentence, and then the second prompt instruction is input into the large language model to obtain the candidate retrieval results output by the large language model as negative retrieval results. The general understanding ability of the large language model can be used to identify the candidate retrieval results related to the first sample sentence, which helps to improve the screening accuracy of the negative retrieval results.

[0031] In a specific implementation scenario, in order to improve the accuracy of the positive example retrieval result, the third retrieval result that is located before the second rank (such as the top 10) when searching based on the first sample sentence can be selected as the positive example retrieval result. It should be noted that the second rank is before the first rank. In the above method, the third retrieval result that is located before the second rank when searching based on the first sample sentence is selected as the positive example retrieval result can improve the accuracy of the positive example retrieval result.

[0032] In one implementation scenario, after obtaining the positive example retrieval results and the negative example retrieval results, the intelligent dialogue model can be trained. It should be noted that the intelligent dialogue model can output the second sample sentence as a training target. Specifically, the first sample sentence and several sample retrieval results can be input into the intelligent dialogue model, and the output sentence of the intelligent dialogue model can be obtained as the third sample sentence that predicts the reply to the first sample sentence, and then the network parameters of the intelligent dialogue model are adjusted based on at least the difference between the second sample sentence and the third sample sentence. As a possible example, in the process of the intelligent dialogue model outputting the third sample sentence, the corresponding decoding probability (that is, the predicted probability value of the decoded character) can be obtained each time it is decoded. On this basis, the decoding probability and the second sample sentence can be combined to perform loss measurement (such as using a loss function such as cross entropy for measurement), and the training loss can be obtained, and then the network parameters of the intelligent dialogue model can be adjusted based on the training loss. In the above method, a first sample sentence and several sample retrieval results are input into the intelligent dialogue model, and an output sentence of the intelligent dialogue model is obtained as a third sample sentence that predicts a reply to the first sample sentence. Then, based at least on the difference between the second sample sentence and the third sample sentence, the network parameters of the intelligent dialogue model are adjusted. The second sample sentence can be output as a training target to force the intelligent dialogue model to learn to select retrieval results related to the question when generating an answer.

[0033] In a specific implementation scenario, several sample retrieval results are also respectively marked with sample labels, and the sample labels represent whether the sample retrieval results are actually related to the first sample sentence and whether they contain the reply answer to the first sample sentence. As a possible example, the sample labels may include at least the following four types: irrelevant, relevant but not containing an answer, relevant and containing a partial answer, and relevant and containing all answers. On this basis, in addition to outputting the second sample sentence as a training target, the intelligent dialogue model can also output the sample labels of each sample retrieval result as a training target. It should be noted that "irrelevant" means that the sample retrieval result is irrelevant to the first sample sentence, "relevant but not containing an answer" means that the sample retrieval result is related to the first sample sentence but does not contain any content that can be used to reply to the first sample sentence, "relevant and containing a partial answer" means that the sample retrieval result is related to the first sample sentence and contains part of the content that can be used to reply to the first sample sentence, and "relevant and containing all answers" means that the sample retrieval result is related to the first sample sentence and contains all the content that can be used to reply to the first sample sentence. In addition, the sample labels can be manually annotated or public data sets can be used, and the source of the sample labels is not limited here. Of course, the above sample labels are only several possible examples in the actual application process, and the specific types of sample labels are not limited here.

[0034] In a specific implementation scenario, when the intelligent dialogue model also outputs the sample labels of each sample retrieval result as a training target, the output label of the intelligent dialogue model for the sample retrieval result can be obtained as the predicted label of the sample retrieval result, and the predicted label represents whether the sample retrieval result is related to the first sample sentence and whether it contains the reply answer of the first sample sentence after prediction. It should be noted that the specific meaning of the predicted label can refer to the relevant description of the aforementioned sample label, which will not be repeated here. On this basis, the network parameters of the intelligent dialogue model can be adjusted based on the difference between the second sample sentence and the third sample sentence, and the difference between the sample label and the predicted label. Exemplarily, the difference between the second sample sentence and the third sample sentence can be measured based on a loss function such as cross entropy to obtain a first training loss (specifically, refer to the relevant description of the aforementioned "training loss"), and the difference between the sample label and the predicted label can be measured based on a loss function such as cross entropy to obtain a second training loss (for example, the probability value of the sample retrieval result belonging to various correlation labels can be predicted, so that the second training loss can be measured based on the sample label and the probability value of various correlation labels), and then the network parameters of the intelligent dialogue model are adjusted based on the first training loss and the second training loss. In addition, it should be noted that the training assisted by relevance labels is not limited to the first sample sentence and its sample retrieval results. Sample question texts different from the first sample sentence and several sample retrieval results of the sample question texts can also be used. The sample question text can correspond to one sample retrieval result, or the sample question text can correspond to multiple sample retrieval results, which are not limited here. The above method uses relevance labels to assist the training of the intelligent dialogue model, further strengthening the learning to select retrieval results related to the question when generating answers, which can improve the interpretability of the intelligent dialogue model.

[0035] In a specific implementation scenario, please refer to Figure 2b , Figure 2b FIG. 1 is a schematic diagram of a process for training an embodiment of an intelligent dialogue model. Figure 2bAs shown, still taking the first sample sentence "Is Province B suitable for the elderly to go in winter?" as an example, its sample retrieval results may include negative retrieval results: "The average temperature in Province C is above 20 degrees in winter...", "Province C is the peak season for northerners to travel south in winter", "According to statistics, the number of elderly people traveling in Province B has reached a new high..." and positive retrieval results "Province B is extremely cold in winter, and the temperature is extremely low. People with cardiovascular and cerebrovascular diseases and the elderly are not suitable to travel to Province B in winter, but it is suitable to go there in summer to escape the heat." The sample labels of the above sample retrieval results are: irrelevant, irrelevant, relevant but not containing an answer, relevant and containing all answers. On this basis, the intelligent dialogue model can output a third sample sentence that predicts a reply to the first sample sentence, "Province B is extremely cold in winter, so it is not suitable for the elderly to go to Province B in winter", as well as the predicted labels of the above sample retrieval results (such as irrelevant, irrelevant, relevant but not containing an answer, relevant and containing all answers). On this basis, the training loss can be calculated by referring to the aforementioned related description, and the network parameters of the intelligent dialogue model can be adjusted accordingly. Of course, Figure 2b The example shown is only one possible example in the actual application process, and other possible situations will not be given one by one here. In addition, after the intelligent dialogue model training converges, the intelligent dialogue model can be used in the reasoning application stage to combine the first sentence and its several search results to generate a second sentence for replying to the first sentence.

[0036] In the above scheme, a search is performed based on the first sentence to be replied to, and several search results are obtained. The first sentence and the several search results are then input into the intelligent dialogue model to obtain the output sentence of the intelligent dialogue model as the second sentence to reply to the first sentence. The intelligent dialogue model is trained based on sample data. The sample data includes: a first sample sentence, a second sample sentence that correctly answers the first sample sentence, and several sample search results of the first sample sentence. The several sample search results at least include positive search results based on the first sample sentence search and negative search results based on the sample negative sentence search. The sample negative sentence is obtained by replacing the entity in the first sample sentence. Therefore, by replacing the entity in the first sample sentence as the sample negative sentence, a negative search result that is extremely interfering with the intelligent dialogue model when answering the first sample sentence can be obtained based on the sample negative sentence. Therefore, during the training process, the intelligent dialogue model can use the second sample sentence that correctly answers the first sample sentence as the output target, learn to accurately distinguish between positive search results that are helpful in answering the input question and negative search results that are not helpful in answering the input question, and alleviate the model's incorrect reference problem as much as possible, thereby improving the accuracy of the model's answer in the retrieval enhancement scenario.

[0037] See also Figure 3 , Figure 3It is a schematic diagram of the framework of an embodiment of the intelligent question-answering device of the present application. The intelligent question-answering device 30 includes: a question retrieval module 31 and a question answering module 32. The question retrieval module 31 is used to search based on the first sentence to be replied to and obtain a number of search results; the question answering module 32 is used to input the first sentence and the number of search results into the intelligent dialogue model, and obtain the output sentence of the intelligent dialogue model as the second sentence to reply to the first sentence; wherein the intelligent dialogue model is trained based on sample data, and the sample data includes: a first sample sentence, a second sample sentence that correctly replies to the first sample sentence, and a number of sample search results of the first sample sentence, and the number of sample search results at least include a positive search result based on the first sample sentence and a negative search result based on the sample negative sentence, and the sample negative sentence is obtained by replacing the entity in the first sample sentence.

[0038] In the above scheme, the intelligent question-answering device 30 performs a search based on the first sentence to be answered, obtains a number of search results, and then inputs the first sentence and the number of search results into the intelligent dialogue model to obtain an output sentence of the intelligent dialogue model as the second sentence to answer the first sentence. The intelligent dialogue model is trained based on sample data, and the sample data includes: a first sample sentence, a second sample sentence that correctly answers the first sample sentence, and a number of sample search results of the first sample sentence. The number of sample search results at least include a positive search result based on the first sample sentence and a negative search result based on the sample negative sentence. The sample negative sentence is obtained by replacing the entity in the first sample sentence. Therefore, by replacing the entity in the first sample sentence as the sample negative sentence, a negative search result that is extremely interfering with the intelligent dialogue model when answering the first sample sentence can be obtained based on the sample negative sentence, so that the intelligent dialogue model can use the second sample sentence that correctly answers the first sample sentence as the output target during the training process, learn to accurately distinguish between positive search results that are helpful in answering the input question and negative search results that are not helpful in answering the input question, and alleviate the problem of model misreference as much as possible, thereby improving the accuracy of the model answer in the retrieval enhancement scenario.

[0039] In some disclosed embodiments, the intelligent question and answer device 30 includes an entity selection module, which is used to select each entity in the first sample sentence as a target entity; the intelligent question and answer device 30 includes an instruction construction module, which is used to construct a first prompt instruction based on the first sample sentence and the target entity; wherein the first prompt instruction is used to instruct the large language model to replace the target entity in the first sample sentence; the intelligent question and answer device 30 includes a negative example acquisition module, which is used to input the first prompt instruction into the large language model to obtain the output sentence of the large language model as a sample negative example sentence.

[0040] In some disclosed embodiments, the intelligent question and answer device 30 includes a first selection module for selecting each first retrieval result based on the sample negative example sentence retrieval as a first negative example result; the intelligent question and answer device 30 includes a candidate acquisition module for obtaining a number of candidate retrieval results based at least on the first negative example result; the intelligent question and answer device 30 includes a candidate screening module for selecting a candidate retrieval result as a negative example retrieval result based on the correlation between the first sample sentence and each candidate retrieval result.

[0041] In some disclosed embodiments, the intelligent question-and-answer device 30 includes a second selection module for selecting a second search result located after the first rank when searching based on the first sample sentence as a second negative example result; the candidate acquisition module includes combining the first negative example result and the second negative example result to obtain several candidate search results.

[0042] In some disclosed embodiments, the candidate screening module includes a construction submodule for constructing a second prompt instruction based on a first sample sentence and a number of candidate retrieval results; wherein the second prompt instruction is used to instruct the large language model to filter out candidate retrieval results related to the first sample sentence; the candidate screening module includes an acquisition submodule for inputting the second prompt instruction into the large language model to obtain the candidate retrieval results output by the large language model as negative example retrieval results.

[0043] In some disclosed embodiments, the intelligent question-answering device 30 includes a third selection module for selecting a third search result that is located before the second rank when searching based on the first sample sentence as a positive search result.

[0044] In some disclosed embodiments, the intelligent dialogue model outputs a second sample sentence as a training target, and the intelligent question and answer device 30 includes a reply prediction module for inputting a first sample sentence and a plurality of sample retrieval results into the intelligent dialogue model, and obtaining an output sentence of the intelligent dialogue model as a third sample sentence that predicts a reply to the first sample sentence; the intelligent question and answer device 30 includes a parameter adjustment module for adjusting the network parameters of the intelligent dialogue model based at least on the difference between the second sample sentence and the third sample sentence.

[0045] In some disclosed embodiments, several sample retrieval results are also respectively marked with sample labels, and the sample labels represent whether the sample retrieval results are actually related to the first sample sentence and whether they include a reply answer to the first sample sentence. The intelligent question and answer device 30 includes a label prediction module, which is used to obtain the output label of the intelligent dialogue model for the sample retrieval results as a predicted label for the sample retrieval results; wherein the predicted label represents whether the sample retrieval results are predicted to be related to the first sample sentence and whether they include a reply answer to the first sample sentence; the parameter adjustment module is specifically used to adjust the network parameters of the intelligent dialogue model based on the difference between the second sample sentence and the third sample sentence, and the difference between the sample label and the predicted label.

[0046] In some disclosed embodiments, the sample labels include at least the following four types: irrelevant, relevant but not including the answer, relevant and including part of the answer, and relevant and including the whole answer.

[0047] See also Figure 4 , Figure 4 : is a schematic diagram of the framework of an embodiment of an electronic device of the present application. The electronic device 40 includes at least a memory 41 and a processor 42 coupled to each other, the memory 41 stores at least program instructions, and the processor 42 is used to execute the program instructions to implement the steps in any of the above-mentioned intelligent question-answering method embodiments. For details, please refer to the aforementioned disclosed embodiments, which will not be repeated here. As a possible example, the electronic device 40 may include but is not limited to a server, a camera, a smart phone, a tablet computer, etc., and the specific type of the electronic device 40 is not limited here.

[0048] Specifically, the processor 42 is used to control itself and the memory 41 to implement the steps in any of the above-mentioned intelligent question-answering method embodiments. The processor 42 can also be called a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip with signal processing capabilities. The processor 42 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 42 can be implemented by an integrated circuit chip.

[0049] In the above scheme, the electronic device 40 performs a search based on the first sentence to be replied to, obtains a number of search results, and then inputs the first sentence and the number of search results into the intelligent dialogue model to obtain an output sentence of the intelligent dialogue model as a second sentence to reply to the first sentence. The intelligent dialogue model is trained based on sample data, and the sample data includes: a first sample sentence, a second sample sentence that correctly answers the first sample sentence, and a number of sample search results of the first sample sentence. The number of sample search results at least include a positive search result based on the first sample sentence and a negative search result based on the sample negative sentence. The sample negative sentence is obtained by replacing the entity in the first sample sentence. Therefore, by replacing the entity in the first sample sentence as the sample negative sentence, a negative search result that is extremely interfering with the intelligent dialogue model when answering the first sample sentence can be obtained based on the sample negative sentence, so that the intelligent dialogue model can use the second sample sentence that correctly answers the first sample sentence as the output target during the training process, learn to accurately distinguish between positive search results that are helpful in answering the input question and negative search results that are not helpful in answering the input question, and alleviate the model's incorrect reference problem as much as possible, thereby being able to improve the accuracy of the model's answer in the retrieval enhancement scenario.

[0050] See also Figure 5 , Figure 5 Schematic diagram of a framework of an embodiment of a computer-readable storage medium 50 of the present application. The computer-readable storage medium 50 stores program instructions 51 that can be executed by a processor, and the program instructions 51 are used to implement the steps in any of the above-mentioned intelligent question-answering method embodiments.

[0051] In the above scheme, the computer-readable storage medium 50 performs a search based on the first sentence to be replied to, obtains a number of search results, and then inputs the first sentence and the number of search results into the intelligent dialogue model to obtain an output sentence of the intelligent dialogue model as a second sentence to reply to the first sentence. The intelligent dialogue model is trained based on sample data, and the sample data includes: a first sample sentence, a second sample sentence that correctly answers the first sample sentence, and a number of sample search results of the first sample sentence. The number of sample search results at least include a positive search result based on the first sample sentence and a negative search result based on the sample negative sentence. The sample negative sentence is obtained by replacing the entity in the first sample sentence. Therefore, by replacing the entity in the first sample sentence as the sample negative sentence, a negative search result that is extremely interfering with the intelligent dialogue model when answering the first sample sentence can be obtained based on the sample negative sentence, so that the intelligent dialogue model can use the second sample sentence that correctly answers the first sample sentence as the output target during the training process, learn to accurately distinguish between positive search results that are helpful in answering the input question and negative search results that are not helpful in answering the input question, and alleviate the model's incorrect reference problem as much as possible, thereby being able to improve the accuracy of the model's answer in the retrieval enhancement scenario.

[0052] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0053] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0054] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0055] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0056] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0057] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0058] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. An intelligent question-answering method, characterized in that: include: Perform a search based on the first sentence to be replied to, and obtain a number of search results; Inputting the first sentence and the plurality of search results into an intelligent dialogue model, and obtaining an output sentence of the intelligent dialogue model as a second sentence in reply to the first sentence; Wherein, the intelligent dialogue model is trained based on sample data, and the sample data includes: a first sample sentence, a second sample sentence that correctly responds to the first sample sentence, and several sample retrieval results of the first sample sentence, and the several sample retrieval results at least include positive retrieval results based on the first sample sentence retrieval and negative retrieval results based on the sample negative sentence retrieval, and the sample negative sentence is obtained by replacing the entity in the first sample sentence.

2. The method according to claim 1, characterized in that The step of obtaining the sample negative sentence includes: Selecting each of the entities in the first sample sentence as a target entity; Based on the first sample sentence and the target entity, construct a first prompt instruction; wherein the first prompt instruction is used to instruct the large language model to replace the target entity in the first sample sentence; The first prompt instruction is input to the large language model to obtain an output sentence of the large language model as the sample negative sentence.

3. The method according to claim 1, characterized in that The steps of obtaining the plurality of sample search results include: Selecting each first search result based on the sample negative sentence search as a first negative result; Based at least on the first negative example result, obtain a number of candidate search results; Based on the correlation between the first sample sentence and each of the candidate retrieval results, the candidate retrieval result is selected as the negative example retrieval result.

4. The method according to claim 3, characterized in that Before obtaining a plurality of candidate search results based at least on the first negative example result, the method further includes: Selecting a second search result that is located after the first rank when searching based on the first sample sentence as a second negative example result; The step of obtaining a plurality of candidate search results based at least on the first negative example result includes: The plurality of candidate retrieval results are obtained by combining the first negative example result and the second negative example result.

5. The method according to claim 3, characterized in that: The selecting the candidate retrieval result as the negative retrieval result based on the correlation between the first sample sentence and each of the candidate retrieval results respectively includes: Based on the first sample sentence and the plurality of candidate search results, construct a second prompt instruction; wherein the second prompt instruction is used to instruct the large language model to filter out the candidate search results related to the first sample sentence; The second prompt instruction is input to the large language model to obtain a candidate retrieval result output by the large language model as the negative example retrieval result.

6. The method according to claim 1 or 3, characterized in that: The step of obtaining the plurality of sample search results further includes: The third search result that is located before the second rank when searching based on the first sample sentence is selected as the positive example search result.

7. The method according to claim 1, characterized in that The intelligent dialogue model outputs the second sample sentence as a training target, and the training steps of the intelligent dialogue model include: Inputting the first sample sentence and the plurality of sample search results into the intelligent dialogue model, and obtaining an output sentence of the intelligent dialogue model as a third sample sentence for predicting a reply to the first sample sentence; Based at least on the difference between the second sample sentence and the third sample sentence, the network parameters of the intelligent dialogue model are adjusted.

8. The method according to claim 7, characterized in that The plurality of sample retrieval results are also respectively annotated with sample labels, wherein the sample labels indicate whether the sample retrieval results are actually related to the first sample sentence and whether they contain a reply answer to the first sample sentence. Before adjusting the network parameters of the intelligent dialogue model based at least on the difference between the second sample sentence and the third sample sentence, the method further includes: Obtaining an output label of the intelligent dialogue model for the sample retrieval result as a predicted label of the sample retrieval result; wherein the predicted label indicates whether the sample retrieval result is predicted to be related to the first sample sentence and whether it contains a reply answer to the first sample sentence; The adjusting the network parameters of the intelligent dialogue model based at least on the difference between the second sample sentence and the third sample sentence includes: Based on the difference between the second sample sentence and the third sample sentence, and the difference between the sample label and the predicted label, the network parameters of the intelligent dialogue model are adjusted.

9. The method according to claim 8, characterized in that The sample labels include at least the following four types: irrelevant, relevant but not including the answer, relevant and including part of the answer, and relevant and including all the answers.

10. An intelligent question-answering device, characterized in that: include: A question retrieval module, used to perform a search based on the first sentence to be answered, and obtain a number of search results; a question answering module, configured to input the first sentence and the plurality of search results into an intelligent dialogue model, and obtain an output sentence of the intelligent dialogue model as a second sentence in reply to the first sentence; Wherein, the intelligent dialogue model is trained based on sample data, and the sample data includes: a first sample sentence, a second sample sentence that correctly responds to the first sample sentence, and several sample retrieval results of the first sample sentence, and the several sample retrieval results at least include positive retrieval results based on the first sample sentence retrieval and negative retrieval results based on the sample negative sentence retrieval, and the sample negative sentence is obtained by replacing the entity in the first sample sentence.

11. An electronic device, characterized in that: The intelligent question-answering method comprises at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the intelligent question-answering method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the intelligent question-answering method according to any one of claims 1 to 9.

Images