Question and answer method and training sample generation method for question and answer

Abstract

The invention provides a question answering method and a training sample generation method. The question-answering method comprises the steps of obtaining a query instruction; executing a cyclic process through the single target model until a preset condition is met, the cyclic process comprising: generating reasoning information based on a preset system prompt instruction, a query instruction and current auxiliary information; generating tool calling information based on the reasoning information; if the tool calling information is the first tool calling information, calling a search tool to search based on the first tool calling information to obtain a search result; wherein the preset condition comprises that the generated tool calling information is second tool calling information; the current auxiliary information is generated based on reasoning information, tool calling information and search results generated in the previous cycle process; and if the tool calling information is second tool calling information, calling a reply model to generate a reply corresponding to the query instruction based on the query instruction and the reasoning information, the tool calling information and the search result obtained in each cycle process.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular to a question and answer method, a training sample generation method and device for question and answer, an electronic device and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, large language models (LLM) can achieve better performance in various tasks due to their built-in massive background knowledge reserves, among which knowledge question and answer has become one of the core application directions. However, in the knowledge question and answer scenario, LLM still faces some challenges, such as timeliness and objectivity. That is, the base knowledge of LLM usually depends on training data, which is difficult to update in real time to cover the latest information, and its output is easily affected by its own knowledge limitations, which may exist bias or information lag. For this problem, if only relying on the base knowledge of LLM itself, it is necessary to build large-scale parameters to realize the storage of massive world knowledge, but large-scale parameters will seriously restrict the reasoning speed and running performance of the model, greatly increase the cost of hardware deployment and computing power consumption, and the effect of its knowledge update is always limited. Therefore, how to realize efficient information search in this scenario has become a technical direction that needs to be focused on. SUMMARY

[0003] This summary is provided to introduce a selection of concepts, which are further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0004] In a first aspect, according to one or more embodiments of the present disclosure, a question and answer method is provided, comprising:

[0005] obtaining a query instruction;

[0006] performing a loop process by a single target model until a preset condition is met, wherein the loop process comprises: generating reasoning information based on a preset system prompt instruction, the query instruction and current auxiliary information; generating tool invocation information based on the reasoning information; if the tool invocation information is first tool invocation information, invoking a search tool to search based on the first tool invocation information to obtain a search result; wherein the preset condition comprises that the generated tool invocation information is second tool invocation information; and the current auxiliary information is generated based on the reasoning information, tool invocation information and search result generated in the last loop process;

[0007] If the tool invocation information is second tool invocation information, a reply model is invoked to generate a reply corresponding to the query instruction based on the query instruction and the reasoning information, the tool invocation information and the search result obtained in each round of the loop process.

[0008] In a second aspect, according to one or more embodiments of the present disclosure, a training sample generation method for question answering is provided, comprising:

[0009] obtaining a query instruction;

[0010] performing a loop process until a preset condition is met, wherein the loop process comprises: generating reasoning information based on a preset system prompt instruction, the query instruction and current auxiliary information using a first large language model; generating tool invocation information based on the reasoning information using a second large language model; if the tool invocation information is first tool invocation information, invoking a search tool to search based on the first tool invocation information to obtain a search result; wherein the preset condition comprises that the generated tool invocation information is second tool invocation information.

[0011] generating training samples for training a target model based on the reasoning information and the tool invocation information generated in the loop process.

[0012] In a third aspect, according to one or more embodiments of the present disclosure, a question answering device is provided, comprising:

[0013] a first obtaining unit configured to obtain a query instruction;

[0014] a first reasoning and searching unit configured to perform a loop process by a single target model until a preset condition is met, wherein the loop process comprises: generating reasoning information based on a preset system prompt instruction, the query instruction and current auxiliary information; generating tool invocation information based on the reasoning information; if the tool invocation information is first tool invocation information, invoking a search tool to search based on the first tool invocation information to obtain a search result; wherein the preset condition comprises that the generated tool invocation information is second tool invocation information; and the current auxiliary information is generated based on the reasoning information, the tool invocation information and the search result generated in the last round of the loop process.

[0015] a reply unit configured to, if the tool invocation information is second tool invocation information, invoke a reply model to generate a reply corresponding to the query instruction based on the query instruction and the reasoning information, the tool invocation information and the search result obtained in each round of the loop process.

[0016] In a fourth aspect, according to one or more embodiments of the present disclosure, a training sample generation device for question answering is provided, comprising:

[0017] a second obtaining unit, configured to obtain a query instruction;

[0018] a second reasoning searching unit, configured to perform a loop process until a preset condition is met, wherein the loop process comprises: generating reasoning information based on a preset system prompt instruction, the query instruction and current auxiliary information by using a first large language model; generating tool invocation information based on the reasoning information by using a second large language model; if the tool invocation information is first tool invocation information, invoking a search tool to search based on the first tool invocation information to obtain a search result; and wherein the preset condition comprises that the generated tool invocation information is second tool invocation information.

[0019] a sample construction unit, configured to generate a training sample for training a target model based on the reasoning information and the tool invocation information generated in the loop process.

[0020] In a fifth aspect, one or more embodiments of the present disclosure provide an electronic device, comprising: at least one memory and at least one processor; wherein the memory is configured to store program code, and the processor is configured to invoke the program code stored in the memory to cause the electronic device to perform the method provided in one or more embodiments of the present disclosure.

[0021] In a sixth aspect, one or more embodiments of the present disclosure provide a non-transitory computer storage medium, which stores program code, and the program code is executed by a computer device to cause the computer device to perform the method provided in one or more embodiments of the present disclosure.

[0022] The question and answer method provided by one or more embodiments of the present disclosure adopts a single model node based on guided reasoning loop iteration search scheme. On the one hand, by adopting a loop iteration process of reasoning and searching performed by a single model node, the workflow of different tasks can follow a unified paradigm, and the search direction and range can be accurately controlled, so as to eliminate the asynchronous error accumulation caused by multi-node cooperation, improve the consistency and search efficiency of task execution, and reduce operation and maintenance cost. On the other hand, by adopting a loop iteration search scheme based on guided reasoning, the content analysis of existing search results can be realized by loop reasoning, and it can be judged whether there is an information gap, and then the tool invocation behavior and the search direction are guided, which improves the resource utilization rate, the fault tolerance rate of low-quality noise search results and the response speed. BRIEF DESCRIPTION OF DRAWINGS

[0023] The above-described and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent as various embodiments of the present disclosure are described in conjunction with the following detailed description, which, taken in conjunction with the drawings, can illustrate certain embodiments of the disclosure. Throughout the drawings, like reference numerals will be understood to refer to like elements, features, and structures. It should be noted that the drawings are not necessarily drawn to scale.

[0024] Figure 1 A flowchart of a question and answer method provided for an embodiment of the present disclosure;

[0025] Figure 2 A flowchart of a training sample generation method for a question and answer provided for an embodiment of the present disclosure;

[0026] Figure 3 An illustration of performing multiple independent samplings in each round of a loop process based on a same query instruction provided for an embodiment of the present disclosure;

[0027] Figure 4 A structural diagram of a question and answer device provided for an embodiment of the present disclosure;

[0028] Figure 5 A structural diagram of a training sample generation device for a question and answer provided for an embodiment of the present disclosure;

[0029] Figure 6 A structural diagram of an electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While several embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. It should be understood that the drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

[0031] It should be understood that the steps recited in the embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this regard.

[0032] As used herein, the term "includes" and its variants are open-ended, and mean "comprises

[0033] The term "determining" encompasses a wide variety of actions and, therefore, "determining" can include calculating, computing, processing, deriving, investigating, looking up (such as looking up in a table, a database or another data structure), ascertaining and the like, and can include receiving (such as receiving information), accessing (such as accessing data in a memory) and the like. The term "determining" can also mean resolving, selecting, choosing, establishing and the like.

[0034] It can be understood that the data involved in the technical solutions of the present disclosure (including but not limited to the data itself, the obtaining or use of the data) should comply with the relevant legal regulations.

[0035] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the use range, the use scenario and the like should be informed to the user and the authorization of the user should be obtained through appropriate means according to the relevant legal regulations. For example, in response to receiving the active request of the user, prompt information is sent to the user to explicitly prompt the user that the operation to be performed by the user will need to obtain and use the personal information of the user, so that the user can voluntarily choose whether to provide the personal information to the software or hardware such as an electronic device, an application program, a server or a storage medium that performs the operation of the technical solution of the present disclosure according to the prompt information.

[0036] As an optional but non-limiting implementation manner, in response to receiving the active request of the user, the manner of sending prompt information to the user can be, for example, the manner of a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0037] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation of the present disclosure, and other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0038] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different objects, devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these objects, devices, modules or units.

[0039] It should be noted that the modification of "one" or "multiple" mentioned in the present disclosure is illustrative and not limiting, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0040] For the purpose of the present disclosure, the phrase "A and / or B" means (A), (B) or (A and B).

[0041] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of these messages or information.

[0042] For the timeliness and objectivity of the question and answer scene, the general solution is to introduce network information search capability, and dynamically use the results of network information to assist question and answer. The inventors found that the related art provides two solutions:

[0043] (1) Solution path one: multiple model nodes (such as model nodes for question disassembly, model nodes for search decision making, and model nodes for result integration) are used for cooperation, and this scheme connects each model node through a specific workflow. However, in this scheme, multiple node models need to cooperate, the maintenance cost is high, and error accumulation problems are easy to occur, and the task splitting on the link and the definition of each node task are inconsistent, related tasks cannot be optimized uniformly, and the search direction depends on multiple node cooperation, which can cause low search quality and large noise of referenced information.

[0044] (2) Solution path two: a single model node directly draws conclusions through a single round of search, and the search quality depends on the immediate decision of a single thinking chain of the model, and lacks multiple rounds of review of whether the search results are complete and whether additional search is needed.

[0045] To solve at least one of the above problems, an embodiment of the present disclosure provides a question and answer method using a single model node based on a guided reasoning cycle iteration search scheme, which refers to Figure 1 , comprising the following steps:

[0046] Step S1, acquiring a query instruction.

[0047] Step S2, generating reasoning information based on the preset system prompt instruction, the query instruction and the current auxiliary information.

[0048] Step S3, generating tool invocation information based on the reasoning information.

[0049] If the tool invocation information generated in step S3 is first tool invocation information, step S4 is executed; if the tool invocation information generated in step S3 is second tool invocation information, step S5 is executed.

[0050] Step S4, invoking a search tool to search based on the first tool invocation information to obtain a search result.

[0051] Next, returning to step S2, steps S2-S4 are executed in a loop until the tool invocation information generated in step S3 is second tool invocation information, and then step S5 is executed. In the loop (i.e., steps S2-S4), a single-target model is executed.

[0052] Step S5, invoking a reply model to generate a reply corresponding to the query instruction based on the query instruction and the reasoning information, the tool invocation information and the search result obtained in each round of loop.

[0053] The current auxiliary information is generated based on the reasoning information, the tool invocation information and the search result generated in the last round of loop. When step S2 is executed for the first time, there is no reasoning information, tool invocation information and search result, so the current auxiliary information is NULL at this time; when step S2 is executed for the i-th time (i>1), the current auxiliary information can be generated based on the reasoning information, the tool invocation information and the search result obtained in the (i-1)-th round of loop. For example, the reasoning information, the tool invocation information and the search result can be spliced to obtain the current auxiliary information.

[0054] On the one hand, by adopting a single model node to execute the loop iteration process of reasoning and search, the workflow of different tasks can follow a unified paradigm, and the search direction and range can be accurately controlled, so that the asynchronous error accumulation caused by multi-node cooperation can be eliminated, the consistency and search efficiency of task execution can be improved, and the operation and maintenance cost can be reduced. On the other hand, by adopting a guided reasoning-based loop iteration search scheme, the content analysis of the existing search result can be realized by loop reasoning, and it can be judged whether there is an information gap, and then the tool invocation behavior and the search direction are guided, and the resource utilization, the fault tolerance of low-quality noise search results and the response speed are improved.

[0055] In some embodiments, the condition for ending the loop process (i.e., the preset condition) can also include that the number of rounds of the executed loop process reaches a preset threshold, in other words, even if the tool invocation information generated in step S3 is not the second tool invocation information, but the number of rounds of the executed loop process reaches the preset threshold, the reply can be generated in step S5, so as to prevent resource waste caused by excessive execution of the loop process.

[0056] In some embodiments, the query instruction includes at least one of the following: an original query instruction input by the user, a rewritten query instruction generated based on the original query instruction. The original query instruction includes but is not limited to an instruction input by the user in the form of text, voice, image, video, sensor signal, etc., and the content thereof can include a question or a prompt sentence guiding the model to perform a specific task. In some embodiments, a user instruction input module can be provided, which is configured to receive multi-modal input content, format-convert different forms of input, and generate query data in a unified structure, so as to facilitate subsequent processing units to perform semantic analysis and response generation. In some embodiments, a multi-modal input interaction component can be provided, which includes a text editing box, a picture uploading control, and a voice collection control. The text editing box is used to receive a text instruction input by the user, the picture uploading control supports import and preview of an image file, and the voice collection control can collect and convert a voice signal into a text format in real time. In some embodiments, natural language processing technology can be used to perform semantic analysis on the query instruction, identify core keywords, and then perform rewriting through methods such as synonym expansion, entity normalization processing, sentence structure adjustment, and context-aware expansion, to generate a rewritten query instruction more suitable for subsequent processing.

[0057] In some embodiments, the query instruction can include a query instruction currently input by the user and one or more historical query instructions input by the user. The historical query instruction is a past query content initiated by the user before the current interaction and related to the current query topic, which can be used to assist the model to understand the continuity and context association of the user query intention.

[0058] In some embodiments, the system prompt instruction comprises role task information and tool calling rule information. The role task information is used to assign a role identity and / or a task to the target model. For example, the system prompt instruction can inform the target model that it is a search assistant for the reply model, and the task is to call one or more tools to assist the user in querying. The tool calling rule information is used to inform the target model of the tools it can call and the usage rules. The content can include tool name description, tool function description, tool parameter requirement, and tool calling format specification. Taking a search tool as an example, the tool name can be “Search”, the tool function description content can be “used to perform search query and return search results”, the tool parameter requirement includes parameter type “object” (object), parameter “query” (search query statement), and the rule of the parameter includes data type array, array element is string (such as search query string), quantity limit is up to N search queries, content requirement is that each query needs to focus on a specific aspect of the original question, and the information overlap degree between queries is as low as possible. The tool calling format specification can be used to specify the output format of tool calling, for example, a JSON object containing the tool name and tool parameter needs to be returned and wrapped in a specific XML tag.

[0059] In some embodiments, the inference information is used to decompose the query instruction into multiple sub-questions and provide corresponding search strategies. For example, in the inference information generated in the first execution of step S2, the content contains a list of sub-questions to be solved and gives the corresponding search strategies. In the inference information generated in the i-th execution of step S2 (i > 1), the content contains analysis of the information already obtained, further decomposition of sub-questions, clear information content still missing, and further search strategies.

[0060] In some embodiments, the first tool calling information comprises a first tool name and a tool parameter thereof; wherein the tool parameter information comprises one or more search statements. For example, the first tool name can be “Search”, and the tool parameter is one or more search query statements generated by the target model based on the query instruction.

[0061] In some embodiments, a search engine interface (Application Programming Interface, API) can be called to search from the Internet and databases based on the tool parameter in the first tool calling information, and the search results are formed into structured data for subsequent processing, such as generating inference information as part of the current auxiliary information in the next round of loop process, or for generating a final reply.

[0062] In some embodiments, the search results can also be filtered, sorted and information extracted. Among them, the filtering process can be used to filter advertisements, repetitions or low-quality content; the sorting process can include sorting search results based on keyword matching, webpage authority, timeliness, etc.; the information extraction process is used to extract the paragraphs or abstracts directly related to the query instruction from the search results, rather than the entire webpage content.

[0063] In some embodiments, the second tool call information can be different from the first tool call information, which can include a tool name different from the first tool name, for example, the second tool call information can include a second tool name "Answer" or other similar names, which are not limited in the present disclosure.

[0064] An embodiment of the present disclosure also provides a training sample generation method for question answering, comprising: obtaining a query instruction; performing a loop process until a preset condition is met, wherein the loop process comprises: using a first large language model to generate reasoning information based on a preset system prompt instruction, the query instruction and current auxiliary information; using a second large language model to generate tool call information based on the reasoning information; if the tool call information is first tool call information, calling a search tool to search based on the first tool call information to obtain search results; wherein the preset condition includes that the generated tool call information is second tool call information; generating training samples for training a target model based on the reasoning information and tool call information generated in the loop process.

[0065] Among them, the first large language model and the second large language model are different types of models, and the target model is a lightweight model relative to the first and second large language models. The inventors have found that if a large language model is directly used to perform the above tasks, it is easy to generate training samples with too long content, and the reasoning content is too exposed to rule information (such as tool call information format, length, search direction restrictions, etc.). To this end, the present embodiment adopts different types of large language models to divide the work, with the first large language model responsible for generating reasoning information in the training sample, and the second large language model responsible for generating tool call information in the training sample, which can make the generated training sample meet the training requirements of the target model, so that the target model can better adapt to the task mode of iterative reasoning and searching. In addition, by using the hard label knowledge distillation scheme, the precise semantic distillation of the large model thinking chain can be realized, and the performance of the lightweight target model can be improved.

[0066] Reference Figure 2The prompt statement (Prompt) can be input to the first large language model to obtain inference information, wherein the prompt statement can include system prompt instructions, query instructions and current auxiliary information. The inference information is input to the second large language model to obtain tool calling information; it is judged whether the tool calling information is the first tool calling information, if not, the process is ended, if yes, the search engine is called to search based on the search query statement in the first tool calling information to obtain search results; the round result of the current round of circulation is obtained based on the search results, the inference information and the search query statement, which is input to the first large language model as the current auxiliary information in the prompt statement for the next round of circulation until the tool calling information is not the first tool calling information, and the process is terminated.

[0067] In some embodiments, in generating the training sample for training the target model based on the inference information and the tool calling information generated in the circulation process, the inference information and the tool calling information are arranged in the order of their generation. For example, the inference information and the tool calling information generated in the ith round are located before the inference information and the tool calling information generated in the i+1th round, and in the same round, the inference information is located before the tool calling information. In this way, the finally generated training sample can be suitable for the guided inference based circulation iteration search task.

[0068] In some embodiments, in each round of the circulation process, at least two independent samplings are performed for the task processing action of the current round, so that the generated training sample can cover the situation of generating answers depending on different search times.

[0069] Referring to Figure 3, which shows that 3 independent samplings are performed in each round of the loop process based on the same query instruction until all path branches obtain the second tool invocation information (i.e., indicating that the reply can be generated). Among them, UQ represents the query instruction, S represents the case that the tool invocation information generated in the current round of the loop process is the first tool invocation information, and A represents the case that the tool invocation information generated in the current round of the loop process is the second tool invocation information. For example, one round of the loop process is performed on the query instruction UQ and 3 samplings are performed, obtaining three cases S10, S11 and A12. Among them, S10 and S11 indicate that the tool invocation information generated in the current round of the loop process is the first tool invocation information, i.e., the search operation needs to be performed in the current round of the loop process, while A12 indicates that the reply can be directly generated without searching in the current round of the loop process. It can be understood that although S10 and S11 both indicate that the tool invocation information generated in the current round of the loop process is the first tool invocation information, S10 and S11 are independently sampled, respectively, and the corresponding reasoning information, tool invocation information and search result are not exactly the same, and S10 and S11 also belong to different path branches. Next, the processing results of cases S10 and S11 perform a second round of the loop process, and also each sample 3 times, thereby obtaining 6 cases S200, A201, S202, A210, S211 and A212. Among them, cases A201, A210 and A212 indicate that the reply can be generated, and cases S200, S202 and S211 perform a third round of the loop process, and each sample 3 times, respectively, obtaining nine cases, i.e., A3000, A3001, A3002, A3020, A3021, A3022, A3110, A3111 and A3112, all of which indicate that the reply can be generated.

[0070] In this example, based on the same query instruction sample UQ, 13 groups of different training samples are generated, including training samples that do not need to search to generate a reply (corresponding to case A12), training samples that perform one search to generate a reply (corresponding to cases A201, A210 and A212), and training samples that perform two searches to generate a reply (corresponding to cases A3000, A3001, A3002, A3020, A3021, A3022, A3110, A3111 and A3112), so that the training samples corresponding to the same query instruction sample can cover the cases of generating a reply depending on different search times, thereby guaranteeing the sample richness.

[0071] Figure 3It is also shown that the training samples are evaluated, where P represents that the training sample passes the evaluation, and F represents that the training sample fails to pass the evaluation. In some embodiments, the training samples can be evaluated based on at least one of the following: whether the inference information and the tool call information meet the formal requirements, the relevance of the inference information to the query instruction, the relevance of the inference information to the tool call information, and the repetition degree of search query statements in tool call information between different rounds. Wherein, evaluating whether the inference information and the tool call information meet the formal requirements includes evaluating whether the inference information and the tool call information meet the pre-defined format specification and character requirements. By evaluating whether the formal requirements are met, it can be ensured that the cyclic iteration process of reasoning and searching is normally executed. Evaluating the relevance of the inference information to the query instruction includes judging whether the subsequent action direction (for example, whether to search) mentioned in the inference information and the search strategy are consistent with the relevant parameters in the tool call information, so as to be used to judge whether the inference information can correctly guide the tools and their parameters called in the tool call information. Evaluating the repetition degree of search query statements in tool call information between different rounds is used to avoid repetition or excessive similarity of search content between different rounds, so as to ensure the novelty of the search direction.

[0072] In some embodiments, the actual search results corresponding to each training sample, and the final reply results obtained by relying on more than two rounds of search can also be evaluated to determine whether they can fully and correctly answer the user's question, whether there are information missing, reference error, data illusion and other problems.

[0073] According to one or more embodiments of the present disclosure, by evaluating the training samples, qualified samples can be screened for subsequent model training, and the training effect is guaranteed.

[0074] In some embodiments, the specific evaluation method can use the following means: using a small parameter scale language model to score and predict the training samples to obtain a score value; and analyzing and evaluating the training samples by a regular strategy.

[0075] In some embodiments, for the training samples that fail to pass the above evaluation, they can be discarded in a specific training phase, or not participate in the training phase (such as a supervised fine-tuning phase), or be used as negative samples in a specific training phase, for example, as low-quality reference samples to form a contrast with high-quality samples in a reinforcement learning phase.

[0076] In some embodiments, for the training samples evaluated above, format conversion can be performed first to ensure that the converted training samples meet the input requirements of the data format of the target training framework, and then the converted training samples are input into the training framework to carry out multiple rounds of supervised fine-tuning training; during the supervised fine-tuning training process, for inference information and tool calling information, loss value calculation operations are performed, and the parameter updating process of the training framework is regulated and controlled through loss value feedback, thereby realizing the training optimization of the target model, and enabling the target model to gradually learn and master the expected task execution paradigm.

[0077] Because the training samples contain multiple rounds of inference information, tool calling information and search results, the content of the training samples is relatively long. In some embodiments, sequence parallel technology can be used for training to improve training efficiency and save computing resources.

[0078] Correspondingly, according to Figure 4 , an embodiment of the present disclosure provides a question and answer device 400, comprising:

[0079] A first acquisition unit 401 is configured to acquire a query instruction.

[0080] A first inference searching unit 402 is configured to execute a loop process through a single target model until a preset condition is met, wherein the loop process comprises: generating inference information based on a preset system prompt instruction, the query instruction and current auxiliary information; generating tool calling information based on the inference information; if the tool calling information is first tool calling information, calling a search tool to search based on the first tool calling information to obtain search results; wherein the preset condition comprises that the generated tool calling information is second tool calling information; and the current auxiliary information is generated based on the inference information, the tool calling information and the search results generated in the last loop process.

[0081] A reply unit 403 is configured to, if the tool calling information is second tool calling information, call a reply model to generate a reply corresponding to the query instruction based on the query instruction and the inference information, the tool calling information and the search results obtained in each loop process.

[0082] In some embodiments, the system prompt instruction comprises role task information and tool calling rule information; wherein the role task information is used to assign a role identity and / or a task to the target model, and the tool calling rule information is used to explain to the target model the tools it can call and the usage rules thereof.

[0083] In some embodiments, the content of the tool calling rule information comprises tool name explanation, tool function description, tool parameter requirement and tool calling format specification.

[0084] In some embodiments, the inference information is used to disassemble the query instruction into multiple questions and provide a search strategy.

[0085] In some embodiments, the first tool call information includes a first tool name and a tool parameter thereof; and the tool parameter includes one or more search statements.

[0086] In some embodiments, the second tool call information includes a second tool name.

[0087] In some embodiments, the inference information and the tool call information in the training sample of the target model are respectively generated by a first large language model and a second large language model.

[0088] Reference Figure 5 According to an embodiment of the present disclosure, a training sample generation apparatus 500 for question answering is provided, comprising:

[0089] A second acquisition unit 501 is configured to acquire a query instruction.

[0090] A second inference search unit 502 is configured to perform a loop process until a preset condition is met, wherein the loop process includes: using a first large language model to generate inference information based on a preset system prompt instruction, the query instruction and current auxiliary information; using a second large language model to generate tool call information based on the inference information; if the tool call information is first tool call information, calling a search tool to search based on the first tool call information to obtain a search result; and the preset condition includes that the generated tool call information is second tool call information.

[0091] A sample construction unit 503 is configured to generate a training sample for training a target model based on the inference information and the tool call information generated in the loop process.

[0092] In some embodiments, at least two independent samplings are performed for each round of the loop process, so that the generated training sample can cover the situation of generating an answer depending on different search times.

[0093] In some embodiments, the apparatus 500 further comprises:

[0094] An evaluation unit is configured to evaluate the training sample based on at least one of the following: whether the inference information and the tool call information meet the formal requirements, the relevance of the inference information to the query instruction, the relevance of the inference information to the tool call information, and the repetition degree of search query statements in tool call information between different rounds.

[0095] For the embodiment of the device, since it basically corresponds to the method embodiment, the relevant part can be referred to the part of the method embodiment. The device embodiment described above is only illustrative, and the modules described as separate modules can or can not be separated. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0096] Accordingly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising:

[0097] at least one memory and at least one processor;

[0098] The memory is configured to store program code, and the processor is configured to invoke the program code stored in the memory to cause the electronic device to perform the question and answer method according to one or more embodiments of the present disclosure.

[0099] Accordingly, according to one or more embodiments of the present disclosure, a non-transitory computer storage medium is provided, which stores program code executable by a computer device to cause the computer device to perform the question and answer method according to one or more embodiments of the present disclosure.

[0100] Reference is made below to Figure 6 which shows a structural schematic diagram of an electronic device (such as a terminal device or a server) 800 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure can include, but is not limited to, terminal devices such as extended reality devices (such as head-mounted displays), mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), digital TVs, desktop computers, etc. It should be noted that, Figure 6 The electronic device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0101] As Figure 6 shown, the electronic device 800 can include a processing device (such as a central processor, a graphics processor, etc.) 801, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or programs loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0102] In general, the following devices can be connected to the I / O interface 805: input devices 806, including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 807, including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 808, including, for example, a magnetic tape, a hard disk, and the like; and communication devices 809. The communication devices 809 can allow the electronic device 800 to communicate wirelessly or wired with other devices to exchange data. Although Figure 6 The electronic device 800 is shown with various devices, but it is understood that all of the shown devices are not required to be implemented or present. More or less devices can alternatively be implemented or present.

[0103] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 809, or installed from the storage devices 808, or installed from the ROM 802. When the computer program is executed by the processing devices 801, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0104] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the disclosure, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, in which the computer-readable program code is contained. Such a propagated data signal can take any of a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including, but not limited to, wire, cable, RF (radio frequency), or the like, or any suitable combination of the foregoing.

[0105] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0106] The computer-readable medium described above can be included in the electronic device; or can exist separately from the electronic device and can be accessed via the electronic device.

[0107] The computer-readable medium described above carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the method of the disclosure described above.

[0108] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0109] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0110] The units described in the embodiments of the present disclosure can be implemented by hardware, software, or a combination of hardware and software. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0111] The functions described in this specification can be implemented in part or in whole through one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0112] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] According to one or more embodiments of the present disclosure, a question and answer method is provided, including: obtaining a query instruction; performing a loop process by a single target model until a preset condition is met, wherein the loop process includes: generating reasoning information based on a preset system prompt instruction, the query instruction, and current auxiliary information; generating tool invocation information based on the reasoning information; if the tool invocation information is first tool invocation information, invoking a search tool to search based on the first tool invocation information to obtain a search result; wherein the preset condition includes that the generated tool invocation information is second tool invocation information; the current auxiliary information is generated based on the reasoning information, the tool invocation information, and the search result generated in the last loop process; if the tool invocation information is second tool invocation information, invoking a reply model to generate a reply corresponding to the query instruction based on the query instruction and the reasoning information, the tool invocation information, and the search result obtained in each loop process.

[0114] According to one or more embodiments of the present disclosure, the system prompt instruction includes role task information and tool invocation rule information; wherein the role task information is used to assign a role identity and / or a task to the target model, and the tool invocation rule information is used to explain to the target model the tools it can invoke and the rules for using the tools.

[0115] According to one or more embodiments of the present disclosure, the content of the tool invocation rule information includes tool name explanation, tool function description, tool parameter requirement, and tool invocation format specification.

[0116] According to one or more embodiments of the present disclosure, the reasoning information is used to disassemble the query instruction into multiple questions and provide a search strategy.

[0117] According to one or more embodiments of the present disclosure, the first tool call information comprises a first tool name and a tool parameter thereof; and the tool parameter comprises one or more search statements.

[0118] According to one or more embodiments of the present disclosure, the second tool call information comprises a second tool name.

[0119] According to one or more embodiments of the present disclosure, the inference information and the tool call information in the training sample of the target model are respectively generated by a first large language model and a second large language model.

[0120] According to one or more embodiments of the present disclosure, a training sample generation method for question answering is provided, comprising: obtaining a query instruction; performing a loop process until a preset condition is met, wherein the loop process comprises: using a first large language model to generate inference information based on a preset system prompt instruction, the query instruction and current auxiliary information; using a second large language model to generate tool call information based on the inference information; if the tool call information is first tool call information, calling a search tool to search based on the first tool call information to obtain a search result; wherein the preset condition comprises that the generated tool call information is second tool call information; and generating a training sample for training a target model based on the inference information and the tool call information generated in the loop process.

[0121] According to one or more embodiments of the present disclosure, for each round of the loop process, at least two independent samplings are performed, so that the generated training sample can cover the situation of generating a reply depending on different search times.

[0122] According to one or more embodiments of the present disclosure, the training sample generation further comprises: evaluating the training sample based on at least one of the following: whether the inference information and the tool call information meet the formal requirements, the relevance of the inference information to the query instruction, the relevance of the inference information to the tool call information, and the repetition degree of search query statements in tool call information between different rounds.

[0123] According to one or more embodiments of the present disclosure, a question and answer device is provided, comprising: a first acquisition unit configured to acquire a query instruction; a first reasoning search unit configured to perform a loop process until a preset condition is met by using a single target model, wherein the loop process comprises: generating reasoning information based on a preset system prompt instruction, the query instruction, and current auxiliary information; generating tool invocation information based on the reasoning information; if the tool invocation information is first tool invocation information, invoking a search tool to search based on the first tool invocation information to obtain a search result; wherein the preset condition comprises that the generated tool invocation information is second tool invocation information; the current auxiliary information is generated based on the reasoning information, the tool invocation information, and the search result generated in the last loop process; and a reply unit configured to, if the tool invocation information is second tool invocation information, invoke a reply model to generate a reply corresponding to the query instruction based on the query instruction and the reasoning information, the tool invocation information, and the search result obtained in each loop process.

[0124] According to one or more embodiments of the present disclosure, a training sample generation device for question and answer is provided, comprising: a second acquisition unit configured to acquire a query instruction; a second reasoning search unit configured to perform a loop process until a preset condition is met, wherein the loop process comprises: generating reasoning information based on a preset system prompt instruction, the query instruction, and current auxiliary information using a first large language model; generating tool invocation information based on the reasoning information using a second large language model; if the tool invocation information is first tool invocation information, invoking a search tool to search based on the first tool invocation information to obtain a search result; wherein the preset condition comprises that the generated tool invocation information is second tool invocation information; and a sample construction unit configured to generate training samples for training a target model based on the reasoning information and the tool invocation information generated in the loop process.

[0125] According to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one memory and at least one processor; wherein the memory is configured to store program code, and the processor is configured to invoke the program code stored in the memory to enable the electronic device to perform the method according to one or more embodiments of the present disclosure.

[0126] According to one or more embodiments of the present disclosure, a non-transitory computer storage medium is provided, which stores program code, and the program code is executed by a computer device to enable the computer device to perform the method according to one or more embodiments of the present disclosure.

[0127] The above description merely illustrates the preferred embodiments of the disclosure and a principle for applying the technologies. It is understood by those skilled in the art that the disclosed scope of the disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features with similar functions disclosed in the disclosure (but not limited to) can be formed.

[0128] Further, although operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order illustrated or sequentially. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, although specific implementation details are contained in the above discussion, these should not be construed as limiting the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0129] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A question-answering method, characterized in that: include: Get query instructions; A loop process is executed through a single target model until a preset condition is met, wherein the loop process includes: generating reasoning information based on a preset system prompt instruction, the query instruction and current auxiliary information; generating tool call information based on the reasoning information; if the tool call information is first tool call information, calling a search tool to search based on the first tool call information to obtain search results; wherein the preset condition includes that the generated tool call information is second tool call information; and the current auxiliary information is generated based on the reasoning information, tool call information and search results generated in the previous loop process; If the tool call information is the second tool call information, the call response model generates a response corresponding to the query instruction based on the query instruction and the reasoning information, the tool call information and the search results obtained during each round of loops.

2. The method according to claim 1, characterized in that The system prompt instruction includes role task information and tool calling rule information; The role task information is used to assign role identities and / or tasks to the target model, and the tool calling rule information is used to explain to the target model the tools that can be called and their usage rules.

3. The method according to claim 2, characterized in that The content of the tool calling rule information includes tool name description, tool function description, tool parameter requirements and tool calling format specification.

4. The method according to claim 1, wherein The reasoning information is used to decompose the query instruction into multiple questions and provide a search strategy.

5. The method according to claim 1, wherein The first tool calling information includes a first tool name and its corresponding tool parameters; wherein the tool parameters include one or more search statements.

6. The method according to claim 1, characterized in that The second tool calling information includes a second tool name.

7. The method according to claim 1, characterized in that The reasoning information and tool calling information in the training samples of the target model are generated by the first large language model and the second large language model respectively.

8. A method for generating training samples for question answering, characterized in that: include: Get query instructions; A loop process is executed until a preset condition is satisfied, wherein the loop process includes: using a first large language model to generate reasoning information based on a preset system prompt instruction, the query instruction, and current auxiliary information; using a second large language model to generate tool call information based on the reasoning information; if the tool call information is first tool call information, calling a search tool to search based on the first tool call information to obtain search results; wherein the preset condition includes that the generated tool call information is second tool call information; A training sample for training a target model is generated based on the inference information and tool call information generated during the cycle.

9. The method according to claim 8, characterized in that For each round in the loop, at least two independent samplings are performed so that the generated training samples can cover situations where responses are generated by different search times.

10. The method according to claim 8, characterized in that The method further comprises: The training samples are evaluated based on at least one of the following: whether the reasoning information and the tool call information meet formal requirements, the relevance of the reasoning information to the query instruction, the relevance of the reasoning information to the tool call information, and the degree of repetition of search query statements in the tool call information between different rounds.

11. A question-answering device, characterized in that: include: A first acquiring unit, configured to acquire a query instruction; A first reasoning search unit is configured to execute a loop process through a single target model until a preset condition is satisfied, wherein the loop process includes: generating reasoning information based on a preset system prompt instruction, the query instruction, and current auxiliary information; generating tool call information based on the reasoning information; if the tool call information is first tool call information, calling a search tool to search based on the first tool call information to obtain search results; wherein the preset condition includes that the generated tool call information is second tool call information; and the current auxiliary information is generated based on the reasoning information, tool call information, and search results generated in the previous loop process; A reply unit is used to call a reply model based on the query instruction and the reasoning information, the tool call information and the search results obtained in each round of loops to generate a reply corresponding to the query instruction if the tool call information is the second tool call information.

12. A training sample generation device for question answering, characterized in that: include: A second acquiring unit, configured to acquire a query instruction; a second reasoning search unit configured to execute a loop process until a preset condition is satisfied, wherein the loop process comprises: using a first large language model to generate reasoning information based on a preset system prompt instruction, the query instruction, and current auxiliary information; using a second large language model to generate tool call information based on the reasoning information; if the tool call information is first tool call information, calling a search tool to perform a search based on the first tool call information to obtain search results; wherein the preset condition comprises that the generated tool call information is second tool call information; The sample construction unit is used to generate training samples for training the target model based on the reasoning information and tool call information generated in the cycle process.

13. An electronic device, characterized in that: include: at least one memory and at least one processor; The memory is used to store program codes, and the processor is used to call the program codes stored in the memory to enable the electronic device to execute the method according to any one of claims 1 to 10.

14. A non-transitory computer storage medium, characterized in that The non-transitory computer storage medium stores a program code, and when the program code is executed by a computer device, the computer device executes the method according to any one of claims 1 to 10.

Images