Search task processing method and target scoring model training method

Abstract

The embodiments of this specification provide a search task processing method and a training method for a target scoring model, wherein the search task processing method includes: determining a target node for a target search task, wherein the target node is an answer that needs to be iteratively optimized; determining a scoring result of the target node through a target scoring model, wherein the target scoring model is obtained based on at least one sample node and corresponding label scoring training, and the label scoring is determined based on the correctness reward or execution efficiency reward of the sample node; performing node expansion based on the scoring result of the target node to determine the search result of the target search task. Introducing the correctness reward or execution efficiency reward of the sample node to determine the label score, obtaining the target scoring model through specific label scoring training, and introducing a proprietary target scoring model to score the target node can quickly, stably and accurately evaluate the quality of the target node, improve the efficiency of scoring and ensure the accuracy of scoring.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of artificial intelligence technology, and in particular to a search task processing method and a target scoring model training method. Background Art

[0002] With the rapid development of computer and artificial intelligence technologies, tools based on large model (LLM) generation are becoming increasingly important. Large models often struggle to directly generate optimal answers when reasoning about complex problems. Therefore, Monte Carlo Tree Search (MCTS) can be used to optimize the answers generated by large models. However, issues such as the time-consuming and inaccurate scoring of MCTS limit its practical application. Therefore, a more efficient and accurate solution for handling search tasks is urgently needed. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a search task processing method. One or more embodiments of this specification also relate to a target scoring model training method, a search task processing apparatus, a target scoring model training apparatus, a computing device, an electronic device, a computer-readable storage medium, and a computer program product to address technical deficiencies in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a search task processing method is provided, including:

[0005] Determining a target node of a target search task, wherein the target node is an answer that needs to be iteratively optimized;

[0006] Determining a scoring result of the target node through a target scoring model, wherein the target scoring model is obtained by training based on at least one sample node and a corresponding label score, and the label score is determined based on a correctness reward or an execution efficiency reward of the sample node;

[0007] Node expansion is performed based on the scoring result of the target node to determine the search result of the target search task.

[0008] According to a second aspect of the embodiments of this specification, a method for training a target scoring model is provided, comprising:

[0009] Obtaining a training sample set under a target search task, wherein the training sample set includes at least one sample node and a corresponding label score, wherein the label score is determined based on a correctness reward or an execution efficiency reward of the sample node;

[0010] Based on the sample nodes in the training sample set, obtaining corresponding prediction scores through an initial scoring model;

[0011] Based on the label score of the sample node and the predicted score, the target loss of the initial scoring model is determined, and the initial scoring model is trained based on the target loss to obtain a trained target scoring model, wherein the target scoring model is used to score the target node in the above-mentioned search task processing method.

[0012] According to a third aspect of the embodiments of this specification, a search task processing device is provided, including:

[0013] A first determination module is configured to determine a target node of a target search task, wherein the target node is an answer that needs to be iteratively optimized;

[0014] a scoring module configured to determine a scoring result of the target node using a target scoring model, wherein the target scoring model is trained based on at least one sample node and a corresponding label score, and the label score is determined based on a correctness reward or an execution efficiency reward of the sample node;

[0015] The second determination module is configured to perform node expansion based on the scoring result of the target node to determine the search result of the target search task.

[0016] According to a fourth aspect of the embodiments of this specification, a training device for a target scoring model is provided, comprising:

[0017] an acquisition module configured to acquire a training sample set under a target search task, wherein the training sample set includes at least one sample node and a corresponding label score, and the label score is determined based on a correctness reward or an execution efficiency reward of the sample node;

[0018] A prediction module is configured to obtain corresponding prediction scores based on the sample nodes in the training sample set through an initial scoring model;

[0019] The training module is configured to determine the target loss of the initial scoring model based on the label score of the sample node and the predicted score, train the initial scoring model based on the target loss, and obtain a trained target scoring model, wherein the target scoring model is used to score the target node in the above-mentioned search task processing method.

[0020] According to a fifth aspect of the embodiments of this specification, there is provided a computing device, including:

[0021] memory and processor;

[0022] Among them, the memory is used to store computer programs / instructions, and the processor is used to execute computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the above-mentioned search task processing method or target scoring model training method are implemented.

[0023] According to a sixth aspect of the embodiments of this specification, an electronic device is provided, including:

[0024] The memory and the processor are connected via a bus;

[0025] Among them, the memory is used to store computer programs / instructions, and the processor is used to execute computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the above-mentioned search task processing method or target scoring model training method are implemented.

[0026] According to the seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned search task processing method or target scoring model training method.

[0027] According to an eighth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned search task processing method or target scoring model training method.

[0028] One embodiment of the present specification provides a search task processing method, which determines a target node of a target search task, wherein the target node is an answer that needs to be iteratively optimized; determines a scoring result of the target node through a target scoring model, wherein the target scoring model is obtained based on at least one sample node and corresponding label scoring training, and the label score is determined based on the correctness reward or execution efficiency reward of the sample node; and performs node expansion based on the scoring result of the target node to determine the search result of the target search task.

[0029] One embodiment of the present specification implements the introduction of correctness rewards or execution efficiency rewards for sample nodes, determines the label scores of sample nodes, obtains a target scoring model through specific label scoring training, and scores the target nodes by introducing a proprietary target scoring model, so that the quality of the target nodes can be quickly, stably and accurately evaluated, the efficiency of scoring is improved and the accuracy of scoring is ensured, thereby efficiently and accurately obtaining the search results of the target search task. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of a Monte Carlo tree search process provided by an embodiment of this specification;

[0031] Figure 2 This is a schematic diagram of a single node expansion in a Monte Carlo tree search optimization provided by an embodiment of this specification;

[0032] Figure 3 This is a flowchart of a search task processing method provided by one embodiment of this specification;

[0033] Figure 4 This is another single-node expansion diagram in Monte Carlo tree search optimization provided by an embodiment of this specification;

[0034] Figure 5 This is a flowchart of a method for training a target scoring model provided by one embodiment of this specification;

[0035] Figure 6 This is a flowchart of a search task processing method provided by one embodiment of this specification;

[0036] Figure 7 This is a structural diagram of a search task processing device provided by an embodiment of this specification;

[0037] Figure 8 This is a schematic diagram of the structure of a training device for a target scoring model provided by one embodiment of this specification;

[0038] Figure 9 This is a structural block diagram of a computing device provided by one embodiment of this specification;

[0039] Figure 10 This is a structural block diagram of an electronic device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0040] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0041] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "an," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0042] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0043] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0044] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. A large model can also be called a foundation model. It is pre-trained on a large amount of unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as a large language model (LLM) and a multi-modal pre-training model.

[0045] When large models are used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. Large models can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, they can be applied to computer vision tasks such as visual question answering (VQA), image description (IC, Image Caption), and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0046] First, the terms involved in one or more embodiments of this specification are explained.

[0047] Monte Carlo Tree Search (MCTS): A heuristic tree search algorithm based on a random sampling strategy, it is widely used in decision-making and planning problems, such as game AI (artificial intelligence) and path planning. Its core idea is to repeatedly simulate and evaluate possible action paths, gradually constructing an asymmetric search tree and selecting the optimal strategy.

[0048] Monte Carlo Tree Search Optimization (MCTS Refine): represents the integration of Monte Carlo tree search with large models, abstracting the iterative refinement process of solving the problem into a search tree structure. The nodes on this tree represent different versions of the answer, and the edges represent improvement attempts.

[0049] Reward Model: A key component in reinforcement learning, particularly reinforcement learning from human feedback (RLHF), it quantifies the quality of generated content (e.g., text, code) and guides model optimization. The reward model can score the quality of the model's output (e.g., a piece of code, an answer) and convert it into an optimizable scalar reward signal.

[0050] Refine: Improve and refine existing answers to increase their quality and accuracy.

[0051] MBPP (Measuring Big-Model Programming Proficiency) dataset: This dataset is used to evaluate the programming proficiency of large models. It can assess the code generation capabilities of large models in multi-language programming scenarios. It can be composed of code snippets from multiple open source code repositories, covering a wide range of programming languages, and is designed to simulate real-world programming tasks and scenarios. By providing a variety of programming problems and corresponding solutions (code), MBPP helps to more comprehensively evaluate the performance of models on programming tasks in different languages ​​and at varying levels of difficulty, thereby promoting the development and optimization of code generation models in the field of multi-language programming.

[0052] It should be noted that with the rapid development of artificial intelligence technology, LLM-based code generation tools play an increasingly important role in software development. However, when dealing with complex problems, LLM often finds it difficult to directly obtain satisfactory answers. Therefore, the MCTS method is used to improve the reasoning ability of the model. Among them, the MCTS Refine method has achieved good results, that is, using MCTS to search and optimize the code generated by LLM.

[0053] In one implementation, the operational workflow of MCTS Refine follows the general pattern of the MCTS algorithm. MCTSRefine uses self-reflection-driven self-improvement to refine the answer, and uses its self-rewarding capability to sample rewards for different versions of the answer to achieve iterative refinement of the answer.

[0054] Figure 1 This is a schematic diagram of a Monte Carlo tree search process provided by an embodiment of this specification, such as Figure 1 As shown, the Monte Carlo tree search consists of four phases: selection, expansion, simulation, and back-propagation. Selection begins with the root node (the current board state) and recursively selects the most promising child nodes until an incompletely expanded node is encountered. Expansion expands a new child node if the selected node has not been fully explored (i.e., there are untried moves). Simulation begins with the newly expanded node and randomly simulates games until the end (a win, loss, or draw). Back-propagation propagates the simulation results (win, loss, reward) back to all nodes along the path, updating their statistics. This process is repeated multiple times (e.g., 1000 simulations), ultimately selecting the child node with the most visits or the highest win rate as the actual move.

[0055] Figure 2 This is a schematic diagram of a single node expansion in a Monte Carlo tree search optimization provided by an embodiment of this specification. For any node S0 to be expanded, the LLM is first called to score the node, and the LLM is called to evaluate the node based on the scoring result. Then, the LLM is called to generate a new answer based on the evaluation result as the expanded node S1.

[0056] In the Monte Carlo Tree Search optimization process described above, each node expansion requires three calls to the large model (scoring, evaluation, and generation), resulting in excessively long inference times, especially when dealing with complex problems. Furthermore, because scoring relies directly on the large model, the scoring results can be highly volatile and inconsistent, impacting the MCTS selection strategy and the quality of the final solution. Furthermore, a fixed number of iterative refinement rounds is typically preset, regardless of problem complexity. This results in excessive time consumption for simple problems and potentially insufficient optimization for complex ones.

[0057] Therefore, an embodiment of the present specification provides a search task processing solution, which significantly reduces scoring time and improves scoring stability by introducing a reward model (also known as a target scoring model) to replace LLM for scoring; by replacing the LLM evaluation process with the use of fixed text, one large model call is reduced each time the expansion is performed; and a specific loss formula can be designed so that the reward model can set a threshold to adaptively adjust the search depth, thereby achieving the goal of quickly answering simple questions and deeply optimizing complex problems. By introducing a reward model to optimize the MCTS Refine process, not only can the efficiency and accuracy of scoring be significantly improved, but a threshold judgment mechanism can also be added to achieve efficient and accurate processing of problems of different complexities, so that simple problems can be solved in one go and complex problems can be thought about in depth.

[0058] To address the above technical issues, this specification provides a search task processing method. One or more embodiments of this specification also involve a target scoring model training method, a search task processing apparatus, a target scoring model training apparatus, a computing device, an electronic device, a computer-readable storage medium, and a computer program product, each of which is described in detail in the following embodiments.

[0059] See also Figure 3 , Figure 3 A flowchart of a search task processing method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0060] Step 302: Determine a target node of the target search task, wherein the target node is the answer that needs to be iteratively optimized.

[0061] The embodiments of this specification are applicable to applications, websites, or mini-programs that have search task processing capabilities. Search task processing functionality is implemented on the application, website, or mini-program. For example, a website that deploys a target scoring model can implement search task processing functions such as code search optimization and text search optimization. For another example, a third-party application can implement the corresponding search task processing functionality by calling the deployed target scoring model through an application programming interface (API).

[0062] It should be noted that target search tasks refer to tasks awaiting search optimization, such as code search optimization tasks and text search optimization tasks. Target search tasks can carry initial task data, which is data that requires iterative search optimization, such as initially generated code or initially generated text.

[0063] In actual implementation, the Monte Carlo tree search method can be used to take the initial task data as the initial node in the Monte Carlo tree, and the optimized task data can be obtained by continuous iterative search as the search result of the target search task.

[0064] For example, taking the target search task as a code search optimization task, assuming that the initial code is code 1, code 1 is expanded to obtain code 2, and code 2 is expanded to obtain code 3. When code 3 meets the stopping condition, code 3 is used as the search result of the target search task, which means that the initial code 1 is iteratively optimized to the final code 3.

[0065] In actual implementation, the target node of the target search task can be determined to facilitate subsequent iterative expansion of the target node. This target node is the node to be expanded and can be the answer to be iteratively optimized, such as code or text. Initially, the initial task data carried by the target search task can be used as the target node. Subsequently, based on the upper confidence bound (UCB) of each child node of the current node, the target node to be expanded can be selected from each child node, achieving a balance between exploration (trying actions that have not been fully evaluated) and exploitation (selecting the currently preferred action). The child nodes of the current node are the nodes expanded from the current node.

[0066] Specifically, the UCB value of each child node of the current node can be calculated by the following formula (1), and the child node with the largest UCB value is selected as the target node to be expanded:

[0067] (1)

[0068] in, Refers to the node's cumulative rewards, is the number of node visits, Refers to the number of parent node visits, is the exploration coefficient.

[0069] For example, let's take the code search optimization task as an example. Assume the initial code is code 1. Code 1 is used as the target node and expanded to obtain code 2 and code 3. The UCB values ​​of code 2 and code 3 are calculated. Code 2, which has a higher UCB value, is selected as the target node. Expand again, and so on until the end.

[0070] In an optional implementation of this embodiment, before determining the target node of the target search task, the method further includes:

[0071] For the target generation task of the target scenario, the target generation content corresponding to the target generation task is obtained through the target generation model;

[0072] A corresponding target search task is created based on the target generation content, wherein the target search task is used to update the target generation content.

[0073] Among them, the target scenario refers to the scenario where content generation is to be performed, such as the code generation scenario, the answer text generation scenario, etc.; the target generation model is a generative large model that can generate corresponding answers to input questions. For example, in the code generation scenario, the target generation model can generate corresponding code for the input question; in the answer text generation scenario, the target generation model can generate corresponding answer text for the input question.

[0074] In actual implementation, the target generation task can be fed into the target generation model, which will then output the corresponding target generation content. A corresponding target search task can then be created based on the target generation content. This target generation content generated by the target generation model can then be iteratively searched for further optimization.

[0075] For example, in a code generation scenario, the target generation task carries the question "Generate a piece of code that sorts multiple data items from largest to smallest." The target generation model can generate code 1, which can sort multiple data items from largest to smallest. In this case, code 1 can be used as the initial task data for search optimization. A corresponding target search task is created. Then, using the Monte Carlo tree search method, an iterative search is performed on code 1 to obtain the optimized code. In other words, the target generation content generated by the target generation model serves as the initial task data, which is then iteratively optimized.

[0076] In the embodiments of this specification, after obtaining the target generation content corresponding to the target generation task through the target generation model, a corresponding target search task can be created based on the target generation content, thereby iteratively searching the content generated by the target generation model, updating and optimizing the target generation content of the large model, and improving the reliability of the model generation results.

[0077] In an optional approach, the Monte Carlo tree search method can be used to iteratively search the content generated by the target generation model, and the Monte Carlo tree search method can be used to update and optimize the target generation content of the large model to improve the reliability of the model generation results.

[0078] Step 304: Determine the scoring result of the target node through a target scoring model, wherein the target scoring model is obtained based on at least one sample node and corresponding label scoring training, and the label scoring is determined based on the correctness reward or execution efficiency reward of the sample node.

[0079] Specifically, the target scoring model is a reward model pre-trained based on at least one sample node and the corresponding label score, and is used to score the quality of the target node. The label score is determined based on the correctness reward and / or execution efficiency reward of the sample node. The correctness reward is used to evaluate whether the content of the sample node is correct, and the execution efficiency reward is used to evaluate the processing efficiency of the sample node. For example, the execution efficiency reward can include a time reward and / or a content reward. The time reward refers to the processing time reward of the sample node for the test sample, and the content reward refers to the content reward of the sample node. For example, the shorter the processing time, the higher the execution efficiency, and the higher the time reward score. The shorter the content length of the sample node, the more streamlined the sample node, the more efficient the execution, and the higher the content reward score.

[0080] In actual implementation, MCTS Refine relies directly on a large model for scoring, which can result in significant volatility and inconsistency, impacting MCTS's selection strategy and the quality of the final answer. By designing a scientific reward formula that enables stable and accurate evaluation of answer quality, the reward model outputs a specific score for each answer, rather than simply a relative ranking, ensuring reliable and consistent scoring.

[0081] In this implementation, a target scoring model is introduced to replace the LLM for node scoring, significantly reducing scoring time and improving scoring stability. Furthermore, by comprehensively considering the correctness reward and / or execution efficiency reward for sample nodes, the label scores of the sample nodes are determined and used as sample labels to train the target scoring model, improving the accuracy and applicability of scoring.

[0082] It should be noted that the training process of the target scoring model can be found in the following Figure 5 The contents shown in this specification will not be repeated here.

[0083] Step 306: Node expansion is performed based on the scoring result of the target node to determine the search results of the target search task.

[0084] In actual implementation, based on the scoring results of the target node, the parts of the target node with poor quality can be optimized to obtain the expanded nodes. The target node can be further selected based on the current nodes, and then the process of scoring and expanding the target node can be returned until the stopping condition is met. The node searched by the final iterative search is obtained as the search result of the target search task.

[0085] In one implementation, the scoring results and target node can be input into a generative large model, which can then be instructed to optimize the target node based on the scoring results, output the optimized expanded node, and continue the cycle of selecting the target node, scoring, and expanding. In another implementation, the scoring results can be used to determine whether to continue expanding. If expansion is necessary, the generative large model can output the optimized expanded node; if not, the search can be stopped, and the search results for the target search task can be obtained.

[0086] In an optional implementation of this embodiment, node expansion is performed based on the scoring result of the target node to determine the search results of the target search task, including:

[0087] Determine whether the scoring result of the target node meets the stop search constraint;

[0088] If the stop search constraint is satisfied, the target node is used as the search result of the target search task;

[0089] If the stop search constraint is not satisfied, the target node is expanded to obtain an extended node, and the search results of the target search task are determined based on the current nodes of the target search task, wherein the current nodes of the target search task include the target node and the extended node.

[0090] The stop search constraint refers to a constraint condition configured based on the scoring result of the target node, which is used to control the timing of stopping the search.

[0091] In actual implementation, if the target node's scoring result satisfies the stop search constraint, no further iterative search is performed and the target node is directly used as the search result for the target search task. If the target node's scoring result does not satisfy the stop search constraint, further expansion is required. In this case, the target node can be expanded to obtain an expanded node, and then the search optimization is iterated based on the current nodes of the target search task to determine the search result for the target search task.

[0092] It should be noted that for simple target search tasks, a relatively low number of iterations is sufficient to obtain a high-quality target node, ensuring that the target node's score satisfies the stop-search constraint. For complex target search tasks, a relatively high number of iterations is required to obtain a high-quality target node, ensuring that the target node's score satisfies the stop-search constraint. This means that simple target search tasks require fewer searches, while complex target search tasks require more. In other words, by introducing a check to determine whether the target node's score satisfies the stop-search constraint, we can control the number of searches for target search tasks of varying complexity, avoiding the need for a fixed number of searches for all target search tasks.

[0093] In the embodiments of this specification, a check is introduced to see whether the scoring result of the target node meets the stop search constraint, so that target search tasks of different complexity have different search depths. Simple target search tasks may generate high-quality results after searching once or twice and stop searching. Only complex tasks will search a higher number of times, which improves the reasoning efficiency and has better adaptability.

[0094] In an optional implementation of this embodiment, determining whether the scoring result of the target node satisfies the stop search constraint includes:

[0095] When the score result of the target node is greater than the set score threshold, it is determined that the stop search constraint is satisfied;

[0096] When the scoring result of the target node is less than or equal to the set scoring threshold, it is determined that the stop search constraint is not satisfied.

[0097] It should be noted that the search stop constraint may be that the score result of the target node is greater than a set score threshold, wherein the set score threshold is a pre-set value used to determine whether the quality of the target node is good enough to meet the requirements.

[0098] In actual implementation, if the target node's score is greater than the set score threshold, it indicates that the target node's quality is already good and meets the stop search constraint. The search stops and the target node is used as the search result for the target search task. If the target node's score is less than or equal to the set score threshold, it indicates that the target node's quality is still poor and does not meet the stop search constraint. It is necessary to continue iterative search optimization until the search result for the target search task is obtained.

[0099] In the examples of this specification, the MCTS Refine method presets a fixed number of iterations regardless of the complexity of the problem, resulting in excessive time consumption for simple problems and insufficient optimization for complex problems. A threshold judgment mechanism has been added to output a score based on the target scoring model. When the score reaches the preset threshold, the search is terminated immediately. This adapts to the complexity of the problem and enables the model to distinguish between simple and complex problems, achieving a single pass for simple problems and a deep search for complex problems.

[0100] In an optional implementation of this embodiment, the target node generates target content; and the target node is expanded to obtain an extended node, including:

[0101] Based on the set evaluation prompt words, the target generation model is used to generate the extended content corresponding to the target generation content, and the extended content is used as the extended node.

[0102] The evaluation prompt is a pre-configured fixed text that prompts the target generation model to generate corresponding extended content, which is the optimized content. For example, the evaluation prompt may be "Wait a moment, there seems to be some errors here."

[0103] In actual implementation, the target generation content and the set evaluation prompt words are input into the target generation model. The target generation model modifies the input target generation content and outputs the corresponding extended content as the extended node of the target node.

[0104] It's important to note that during the target generation model's processing, it meticulously analyzes and modifies the target generated content based on the guidance of the pre-set evaluation prompts. For example, if the pre-set evaluation prompts contain errors, the target generation model can attempt to identify and correct the errors in the target generated content. Under the guidance of the pre-set evaluation prompts, the target generation model can generate more accurate and optimized extended content without requiring the large model to provide specific evaluation details, effectively improving the processing performance of the search task.

[0105] In the embodiments of this specification, by replacing the LLM evaluation process with the use of fixed text, one large model call is reduced each time the expansion is performed, thereby further improving the reasoning efficiency of the search task.

[0106] Figure 4 This is another single node expansion diagram in Monte Carlo tree search optimization provided by an embodiment of this specification, such as Figure 4 As shown, for the target node S0, the LLM model can be called for scoring. The scoring result includes the score and evaluation content. Based on the evaluation content, the LLM is called again to generate and obtain the extended node S1. This process still requires calling the large model twice, and the efficiency of extended reasoning is low.

[0107] Taking the target scoring model as the reward model as an example, for the target node S0, the RM (reward model) can be called to score. The scoring result includes a score. Based on the score, the LLM is called to evaluate and obtain the evaluation content. Based on the evaluation content, the LLM is called again to generate and obtain the extended node S1. This process still requires calling the large model twice, and the efficiency of extended reasoning is low.

[0108] For target node S0, the reward model (RM) can be called to score it. The scoring result includes a score, which is directly generated by calling the LLM based on the set evaluation prompt word to obtain the expanded node S1. This process still only requires a single call to the large model, improving the efficiency of expanded reasoning. In addition, a threshold can be set; if the score of target node S0 reaches the threshold, the search will be stopped.

[0109] In an optional implementation of this embodiment, determining the search results of the target search task based on the current nodes of the target search task includes:

[0110] Determine the comprehensive score of each node respectively, determine the updated node to be expanded from each node based on the comprehensive score, use the updated node to be expanded as the updated target node, and return to the step of determining the scoring result of the target node through the target scoring model until the search result of the target search task is obtained.

[0111] The comprehensive score of each node refers to the score calculated by comprehensively considering factors such as the historical score and the number of visits of each node, which can indicate the potential value of each node. For example, the comprehensive score can be the UCB value of each node.

[0112] In actual implementation, the UCB value of each node can be calculated, and the node with the highest UCB value can be selected as the target node to be iteratively expanded. The scoring and expansion process can be iteratively executed until the search result of the target search task is obtained.

[0113] It should be noted that the comprehensive score of each node is taken into consideration to ensure that the node with higher potential value is selected as the target node for the next iteration. The iterative scoring and expansion are continued until the final search result is obtained, which improves the accuracy of node expansion and search.

[0114] An embodiment of the present specification provides a search task processing method, which introduces correctness rewards and / or execution efficiency rewards for sample nodes, determines label scores for sample nodes, obtains a target scoring model through specific label scoring training, and scores target nodes by introducing a proprietary target scoring model, so that the quality of target nodes can be quickly, stably and accurately evaluated, thereby improving the efficiency of scoring and ensuring the accuracy of scoring, thereby efficiently and accurately obtaining search results for target search tasks.

[0115] See also Figure 5 , Figure 5 A flowchart of a method for training a target scoring model according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0116] Step 502: Obtain a training sample set under the target search task, wherein the training sample set includes at least one sample node and a corresponding label score, and the label score is determined based on the correctness reward or execution efficiency reward of the sample node.

[0117] The training sample set for a target search task consists of at least one sample node and the corresponding label score for each sample node. The target search task is a task awaiting search optimization. Sample nodes are multiple versions of content with corresponding label scores. For example, in a code search optimization task, sample nodes are multiple versions of code, while in a text search optimization task, sample nodes are multiple versions of content. Label scores are determined based on the correctness reward or execution efficiency reward of the sample node, which serves as the sample label to train the target scoring model.

[0118] In actual implementation, the training sample set can be obtained from a database or other third-party application platform, or the training sample set can be constructed based on the target search task.

[0119] In an optional implementation of this embodiment, the sample node is a sample answer; obtaining a training sample set for the target search task includes:

[0120] Obtain test examples under the target search task;

[0121] Generate at least one sample answer corresponding to each sample question under the target search task, test each sample answer based on the test sample, and obtain test information of each test sample and each sample answer;

[0122] Determine a label score for each sample answer based on a correctness reward or an execution efficiency reward for each sample answer, wherein the correctness reward or the execution efficiency reward for the sample answer is determined based on test information of the sample answer;

[0123] Based on the answers to each sample and the corresponding label scores, a training sample set for the target search task is obtained.

[0124] Specifically, a test sample refers to data used to test the quality of a sample answer. For example, if the sample answer is a sorting code, the test sample can be a set of values ​​{5, 4, 8, 1, 2, 6, 9}. The test sample is input into the sample answer, and the sample answer can sort the test sample and output the sorting result. The test sample can be used to test whether the sample answer can sort the test sample, whether the sorting result is correct, etc., thereby reflecting the quality of the sample answer.

[0125] In actual implementation, the test samples can be obtained from a database or other third-party platform, or can be constructed using other large models or generation tools.

[0126] It should be noted that at least one sample answer corresponding to each sample question under the target search task can be generated. The sample answer can be generated using other large models or generation tools, or directly obtained from a specific database. For example, taking the code generation scenario as an example, the MBPP dataset can be used as a basis to obtain the sample questions therein, and an open source large model can be used to generate a batch of codes for each sample question in the MBPP dataset and deduplicate them. At least one sample answer corresponding to each sample question can be obtained, thereby ensuring the richness of the sample answers and thus ensuring the model training effect.

[0127] After generating sample answers, each sample answer can be tested using the obtained test examples to obtain test information for each test example and each sample answer. Based on the test information, the test example for the sample answer can be used to test each sample answer, obtaining test information for each test example and each sample answer, and then determining the label score for each sample answer. The test information refers to relevant information about the sample answer's test of the test example, such as the time, whether it passed, and the length of the test example.

[0128] In the embodiments of this specification, test samples under the target search task can be obtained, and sample answers can be generated. Each sample answer is tested based on the test samples to obtain test information of each test sample and each sample answer, thereby determining the correctness reward or execution efficiency reward of each sample answer based on the test information, and then determining the label score of each sample answer, and automatically marking the label score of each sample answer as a sample label, and automatically constructing a training sample set with a label score, that is, constructing a dedicated data set to train a target scoring model, thereby improving the training efficiency and training effect of the target scoring model.

[0129] In an optional implementation of this embodiment, obtaining a test example for a target search task includes:

[0130] Obtain at least one sample question under the target search task and the reference answer to each sample question;

[0131] Generate at least one set of test examples corresponding to each sample problem;

[0132] The test samples are verified based on the reference answers to the sample questions to obtain the test samples that pass the verification.

[0133] It should be noted that in addition to generating sample answers to sample questions as training data, reference answers to sample questions can also be obtained. The reference answers can be high-quality answers and are used to evaluate the quality of test samples, thereby screening out high-quality test samples whose sample answers can be successfully identified and processed, thereby ensuring the quality of the test samples.

[0134] In practical implementation, taking the code generation scenario as an example, the MBPP dataset can be used as a foundation, with sample problems and corresponding code serving as reference answers. Multiple sets of test examples can then be generated for each sample problem using a variety of open-source big models. This enriches the generated test examples. The generated test examples are then verified using MBPP's own code (the reference answers). Passing test examples are removed and retained to form a test sample set.

[0135] In practice, a passed test case refers to one for which the reference answer successfully outputs a result, not necessarily a correct processing result. If multiple reference codes are available for the same sample problem, the test case's passing status can be determined based on the test results of the majority of reference answers across the multiple reference codes.

[0136] As an example, the sample question is "Please generate code that can sort multiple numerical values". The reference answer is code A. Code A can sort multiple input numerical values. The multiple groups of test samples generated for this sample question using multiple open source large models are test sample 1 {5, 4, 8, 1, 2, 6, 9}, test sample 2 {2, 1, 3, 4, 7, 5, 6}, test sample 3 {5, 4, a, 1, 2, 6, 9}, test sample 4 {5, 4, 8, 1, 2, b, 9}, and test sample 5 {5, 4, 8, 1, 2, 6, 9}. Test samples 1-5 are input into code A respectively. For test sample 1, code A outputs {1, 2, 3, 4, 5, 7, 6}; for test sample 2, code A outputs "an error occurred"; for test sample 4, code A outputs "an error occurred"; and for test sample 5, code A outputs {1, 2, 4, 6, 5, 8, 9}. As can be seen from the above, test samples 1, 2, and 5 all output results normally. Even though the sorting results of test samples 2 and 5 are incorrect, they can output results normally. Test samples 1, 2, and 5 all pass the test. In addition, since test sample 5 is repeated with test sample 1, after deduplication, the final test samples obtained are test sample 1 and test sample 2.

[0137] In the embodiments of this specification, reference answers to sample questions can be obtained, the generated test samples can be tested, the test samples that pass the test can be deduplicated, and the final test samples can be obtained to test the sample answers, thereby determining the quality of the sample answers, filtering out erroneous sample data in the generated test samples, and ensuring that the retained test samples are correct test samples that can enable the sample answers to be executed smoothly, thereby ensuring the accuracy of subsequent test sample answers.

[0138] In the embodiments of this specification, the target scoring model may be a reward model. A dedicated dataset training reward model is constructed in the above manner, thereby improving the training efficiency and training effect of the reward model.

[0139] In an optional implementation of this embodiment, the label score of each sample answer is determined based on the correctness reward or execution efficiency reward of each sample answer, including:

[0140] determining, based on the test information, whether a first sample answer has a test example that fails the test, wherein the first sample answer is any one of the sample answers;

[0141] If so, determine the correctness reward of the first sample answer based on the test information, and map the correctness reward of the first sample answer to a first threshold range to obtain a label score for the first sample answer;

[0142] If not, the execution efficiency reward of the first sample answer is determined based on the test information, and the execution efficiency reward of the first sample answer is mapped to the second threshold range to obtain the label score of the first sample answer.

[0143] Among them, the first sample answer is any one of the sample answers. The above judgment is performed on each sample answer to determine whether there are test samples that fail the test, so as to calculate the corresponding label scores in different ways.

[0144] Specifically, the correctness reward is used to evaluate whether the content of the sample answer is correct, and the execution efficiency reward is used to evaluate the processing efficiency of the sample answer. The first threshold range is pre-configured and limits the numerical range of the label score when there are test samples that fail the test; the second threshold range is pre-configured and limits the numerical range of the label score when there are no test samples that fail the test. For example, the first threshold range can be , the second threshold range can be , where λ can be configured based on actual needs.

[0145] It should be noted that when the sample answer does not pass all test examples, it means that for some test examples, the sample answer cannot successfully output results. At this time, only the correctness reward is considered, and the score is mapped to the first threshold range; when the sample answer passes all test examples, it means that for each test example, the sample answer can successfully output results. At this time, the execution efficiency reward of the sample answer is comprehensively considered, such as the execution efficiency reward can include time reward and / or content reward, and the score is mapped to the second threshold range.

[0146] In the embodiments of this specification, a proprietary reward formula is designed to enable the quality of answers to be evaluated stably and accurately, so that the trained target scoring model can output the specific score of each sample answer, rather than just performing relative ranking, thereby ensuring the reliability and consistency of the score output by the target scoring model.

[0147] In an optional implementation of this embodiment, determining the correctness reward for the first sample answer based on the test information includes:

[0148] Obtaining a test result of whether the first sample answer passes the test on each test sample;

[0149] Determining a sample score for each test sample, wherein the sample score for the test sample is determined based on the answer scores of the passed sample answers, the total score of each sample answer, and the test result;

[0150] The correctness reward of the first sample answer is determined based on the sample scores of the test samples passed by the first sample answer, the total scores of the test samples, and the test results.

[0151] In actual implementation, the test result of whether each sample answer passes the test on each test example is a [0,1] matrix, in which the (i, j)th element indicates whether the i-th sample answer passes the j-th test example. If the (i, j)th element is 0, it means that the i-th sample answer fails the j-th test example; if the (i, j)th element is 1, it means that the i-th sample answer passes the j-th test example.

[0152] Specifically, the correctness reward for the first sample answer can be calculated using the following formulas (2)-(4):

[0153] (2)

[0154] (3)

[0155] (4)

[0156] in, represents the code score, Represents the test sample score, M is a 0, 1 matrix indicating whether the code passes the test sample. and is the damping factor. The formula indicates that for a code test, the more correct samples there are, the higher the correct sample score is, and the code score is The higher the score, the more error codes a test set has. The higher.

[0157] In specific implementation, the code score and the test sample score are both initialized to 1, and then the above formulas (3) and (4) are continuously iterated until the code score and the test sample score are stable. A fixed number of iterations can be set, for example, 100 times.

[0158] In addition, M is a 0, 1 matrix that indicates whether the code passes the test sample. M' is used to calculate the test sample score of the above formula (4). For the code, passing will add points, but for the test sample, passing will deduct points. Therefore, the elements "0" and "1" in the 0, 1 matrix that indicates whether the code passes the test sample need to be reversed, which is the above formula (2).

[0159] It should be noted that when the sample answer does not pass all test examples, only the correctness reward is considered, and the score is mapped to the first threshold range, such as , the label score can be calculated by the following formula (5):

[0160] (5)

[0161] The above “λ” can be flexibly configured based on actual needs.

[0162] In the embodiments of this specification, when a sample answer does not pass all test examples, only the correctness reward is considered. The test results of whether the first sample answer passes the test on each test example and the sample scores of each test example are comprehensively considered to determine the correctness reward. A proprietary correctness reward formula is designed to improve the accuracy of the label scores of the labeled sample answers and ensure the model training effect.

[0163] In an optional implementation of this embodiment, the execution efficiency reward includes a time reward and / or a content reward; determining the execution efficiency reward of the first sample answer based on the test information, and mapping the execution efficiency reward of the first sample answer to a second threshold range to obtain a label score for the first sample answer includes:

[0164] Based on the test information, determining the maximum execution time and the minimum execution time of the first sample answer on each test sample, and determining a time bonus score for the first sample answer based on the maximum execution time and the minimum execution time;

[0165] Based on the test information, determine the maximum content length and the minimum content length of each sample answer, and determine the content reward score of the first sample answer;

[0166] Based on the time reward score and / or the content reward score and in combination with a threshold value of the second mapping range, a label score of the first sample answer is determined.

[0167] In actual implementation, the test information may include execution time, content length, etc. When the sample answer passes all test examples, an execution efficiency reward may be considered. The execution efficiency reward may include a time reward and / or a content reward. That is, the time reward / content reward of the sample answer is analyzed considering the execution time of the sample answer passing the test sample, the content length of the sample answer, etc., and the time reward score / content reward score is mapped to the second threshold range to obtain the label score of the sample answer.

[0168] Specifically, the time bonus score (also known as time bonus) is calculated using the following formula (6):

[0169] (6)

[0170] Among them, t max It refers to the maximum execution time of the execution time of each test sample for the first sample answer; tmin refers to the minimum execution time of the execution time of each test sample for the first sample answer, and t refers to the current time.

[0171] The content reward score (also known as length reward) is calculated using the following formula (7):

[0172] (7)

[0173] Among them, I max Refers to the longest content length among the sample answers, I min The shortest content length among the sample answers, and I refers to the current content length. For example, if the sample answer is code, the content length can refer to the number of tokens in the code, or the number of words in the code. For example, the content length of "import pandas" is two words.

[0174] It should be noted that when the sample answer passes all test examples, the time reward and / or content reward of the code are comprehensively considered and the score is mapped to the second threshold range, such as , the label score is calculated by the following formula (8):

[0175] (8)

[0176] The above “λ” can be flexibly configured based on actual needs. Refers to the time bonus score, Refers to the content reward score.

[0177] In the embodiments of this specification, when a sample answer passes all test examples, the time reward and content reward can be comprehensively considered. The maximum execution time and minimum execution time of the sample answer on each test example can be comprehensively considered to determine the time reward score. The maximum content length and minimum content length of each sample answer can be comprehensively considered to determine the content reward score. A proprietary execution efficiency reward formula is designed to improve the accuracy of the label scores of the labeled sample answers and ensure the model training effect.

[0178] Step 504: Based on the sample nodes in the training sample set, obtain corresponding predicted scores through the initial scoring model.

[0179] In actual implementation, the first sample node in the training sample set can be input into the initial scoring model to obtain the predicted score corresponding to the first sample node. The predicted score is the scoring result output by the initial scoring model after analyzing the sample node. The scoring result is the model prediction value, which can be used for subsequent calculation of model loss, and then the initial scoring model is trained.

[0180] In an optional implementation of this embodiment, the method further includes:

[0181] Obtain candidate generative models;

[0182] The language model layer in the candidate generation model is replaced with a linear layer and a mapping layer to obtain an initial scoring model.

[0183] It should be noted that any generative model can be used to construct an initial scoring model, such as the Qwen2.5-Coder-7B-Instruct model.

[0184] Among them, Qwen2.5-Coder-7B-Instruct is an open-source large-scale language model from the Qwen2.5-Coder series, with 7 billion parameters. The series also includes other models with varying parameter sizes to meet diverse user needs. Its large training data scale excels in code generation, reasoning, and repair, with up to 5.5 trillion training tokens. The training data includes source code, text code bases, and synthetic data. Its strong encoding capabilities and support for long contexts facilitate its processing of large code bases.

[0185] Specifically, the generative model needs to generate corresponding content, so the model needs to include a language model layer (lm_head) for content generation. In the embodiments of this specification, the initial scoring model does not need to generate content and only needs to output scores, so the language model layer of the generative model can be removed. In addition, the initial scoring model needs to output scores, so two linear layers and a mapping layer (i.e., Tanh activation function) can be connected to obtain an initial scoring model. The linear layer can be used to extract relevant features, and the mapping layer can be used to map the results to a specific numerical range (such as mapping to 0-1). The generative model is transformed from a generation task (such as code completion) to a scoring task, so that the initial scoring model can be trained to obtain a target scoring model capable of scoring nodes.

[0186] In the embodiments of this specification, any generative model can be used to construct an initial scoring model without building a completely new scoring model, thereby improving the efficiency of model building.

[0187] Step 506: Based on the label scores and predicted scores of the sample nodes, determine the target loss of the initial scoring model, train the initial scoring model based on the target loss, and obtain a trained target scoring model, wherein the target scoring model is used to score the target node in the above-mentioned search task processing method.

[0188] In actual implementation, the model parameters of the initial scoring model can be reversely adjusted based on the target loss. If the training stop condition is not met, the model training is continued based on the next sample answer until the training stop condition is met, and the trained target scoring model is obtained.

[0189] Among them, the label score refers to the result that the initial scoring model actually wants to output, that is, the label score is the actual result, and the sample answer is input into the initial scoring model, and the output prediction score is the prediction result. When the difference between the predicted result and the actual result is small enough, it means that the predicted result is close enough to the actual result. At this time, the initial scoring model training is completed, and it is determined that the training stop condition is met, and the target scoring model is obtained.

[0190] In addition, in addition to judging whether the training stop condition is met based on the loss value, the number of iterations can also be combined to determine whether the training stop condition is met. Specifically, if the target loss is greater than the loss value threshold, it can be further determined whether the number of iterations at this moment has reached the preset number of iterations. If the number of iterations at this moment has not reached the preset number of iterations, it can be determined that the training stop condition has not been met. The model parameters of the initial scoring model can be adjusted, and the model training can be returned to continue based on the next sample answer. When the preset number of iterations is reached, it is determined that the training stop condition is met, and the iteration is stopped to obtain the trained target scoring model.

[0191] Among them, the loss value threshold and the preset number of iterations are set according to actual conditions. The embodiment of the present application does not impose any restrictions on this. When the number of training times reaches the preset number of iterations, it means that the initial scoring model has been trained enough times. At this time, the prediction result of the initial scoring model is very close to the actual result, and the training can be stopped.

[0192] It should be noted that the target loss can intuitively show the difference between the model's predicted results and the actual results. Targeted training of the initial scoring model and adjustment of parameters can effectively improve the model training rate and effect.

[0193] In an optional implementation of this embodiment, determining the target loss of the initial scoring model includes:

[0194] Determine the recall loss of the sample node based on the label score and prediction score of the sample node;

[0195] Based on the recall loss and the set threshold penalty factor, the target loss of the initial scoring model is determined.

[0196] It should be noted that since the specific scores of the sample nodes need to be determined, the ranking loss commonly used in the reward model is discarded and the regression loss is adopted. In addition, in order to ensure the reliability of the threshold, the threshold penalty factor can be further increased. Specifically, the target loss of the initial scoring model is determined by the following formula (9):

[0197] (9)

[0198] in, Score the predictions, Score the tags, is the penalty coefficient, λ is the threshold value pre-configured based on actual needs, is the threshold penalty factor, ensuring that the model is appropriately penalized when the prediction score exceeds the threshold.

[0199] In the embodiments of this specification, by designing a scientific reward formula and a dedicated data set, designing a specific loss formula, and training an initial scoring model, the reward model can adaptively adjust the search depth based on a set threshold, achieving the goal of quickly answering simple questions and deeply optimizing complex problems, thereby improving reasoning efficiency and accuracy.

[0200] The following combined Figure 6 , taking the application of the search task processing method provided in this specification in the code generation scenario as an example, the search task processing method is further explained. Figure 6 A flowchart of a search task processing method provided in an embodiment of this specification is shown, which specifically includes the following steps.

[0201] The task problem carried by the target generation task is "Please generate a piece of code to sort multiple data from large to small". The target generation model can generate code 1, which can sort multiple data from large to small.

[0202] Call the reward model to score Code 1.

[0203] If the score of code 1 exceeds the set threshold, code 1 will be used as the final search result.

[0204] If the score of code 1 does not exceed the set threshold, the target generation model generates code 2 corresponding to code 1 based on the set evaluation prompt word (fixed text). Code 2 optimizes some of the errors in code 1. The Unified Border Crossing (UCB) values ​​of codes 1 and 2 are calculated, and the code with the higher UCB value is selected as the target node. The scoring and expansion steps are repeated again, and so on until the end.

[0205] Next, the search task processing method of the embodiment of this specification is experimentally tested, and the experimental results are as follows:

[0206] Test set: HumanEval

[0207] Test model: Qwen2.5-Coder-32B-Instruct

[0208] The accuracy of the target scoring model (also known as the reward model) is verified as follows:

[0209] Experimental setup: Using Qwen2.5-Coder-32B-Instruct, we sampled five answers for each question in HumanEval and then ranked them using different models. The results are shown in Tables 1 and 2 below. qwen2.5-coder-7b-instruct-Generate-Train uses the same data for generative training with the labels "yes" and "no," using the probability of "yes" as the score. qwen2.5-coder-7b-instruct-RM is a reward model trained using the examples in this manual.

[0210] Table 1 Effect of Qwen2.5-Coder-32B-Instruct generating code

[0211]

[0212] Table 2 The accuracy of the final answer after each model ranks each answer

[0213]

[0214] The performance and overall accuracy of the target scoring model (also known as the reward model) are analyzed as follows:

[0215] Experimental setup: Qwen2.5-Coder-32B-Instruct model, MCTS Refine method set the search depth to 5, RM+MCTS Refine set the maximum depth to 5, and the score threshold to 0.3. The solution using the embodiment of this specification terminated the search early in 85% of scenarios with a termination accuracy of 95.7%, an overall accuracy of 93.2%, an improvement of 3.6%, and a 10x reduction in search time. The average number of LLM calls was optimized from 14 to 1.64 RM calls plus 1.64 LLM calls, and the average search time per question was optimized from 71.1 seconds to 6.4 seconds.

[0216] Table 3 Performance and accuracy table

[0217]

[0218] Corresponding to the above method embodiment, this specification also provides a search task processing device embodiment, Figure 7 FIG. 1 shows a schematic diagram of a search task processing device provided by an embodiment of this specification. Figure 7 As shown, the device includes:

[0219] A first determination module 702 is configured to determine a target node of a target search task, wherein the target node is an answer that needs to be iteratively optimized;

[0220] Scoring module 704 is configured to determine a scoring result for a target node using a target scoring model, wherein the target scoring model is trained based on at least one sample node and a corresponding label score, and the label score is determined based on a correctness reward or an execution efficiency reward for the sample node;

[0221] The second determining module 706 is configured to perform node expansion based on the scoring result of the target node to determine the search result of the target search task.

[0222] Optionally, the second determining module 706 is further configured to:

[0223] Determine whether the scoring result of the target node meets the stop search constraint;

[0224] If the stop search constraint is satisfied, the target node is used as the search result of the target search task;

[0225] If the stop search constraint is not satisfied, the target node is expanded to obtain an extended node, and the search results of the target search task are determined based on the current nodes of the target search task, wherein the current nodes of the target search task include the target node and the extended node.

[0226] Optionally, the target node generates target content; the second determining module 706 is further configured to:

[0227] Based on the set evaluation prompt words, the target generation model is used to generate the extended content corresponding to the target generation content, and the extended content is used as the extended node.

[0228] Optionally, the second determining module 706 is further configured to:

[0229] Determine the comprehensive score of each node respectively, determine the node to be expanded from each node based on the comprehensive score, use the node to be expanded as the updated target node, and return to the step of determining the score result of the target node through the target scoring model until the search result of the target search task is obtained.

[0230] Optionally, the second determining module 706 is further configured to:

[0231] When the score result of the target node is greater than the set score threshold, it is determined that the stop search constraint is satisfied;

[0232] When the scoring result of the target node is less than or equal to the set scoring threshold, it is determined that the stop search constraint is not satisfied.

[0233] Optionally, the device further includes a creation module configured to:

[0234] For the target generation task of the target scenario, the target generation content corresponding to the target generation task is obtained through the target generation model;

[0235] A corresponding target search task is created based on the target generation content, wherein the target search task is used to update the target generation content.

[0236] One embodiment of the present specification provides a search task processing device, which implements the introduction of correctness rewards and / or execution efficiency rewards for sample nodes, determines the label scores of sample nodes, obtains a target scoring model through specific label scoring training, and scores the target nodes by introducing a proprietary target scoring model, so that the quality of the target nodes can be quickly, stably and accurately evaluated, the efficiency of scoring is improved and the accuracy of scoring is ensured, thereby efficiently and accurately obtaining the search results of the target search task.

[0237] The above is a schematic scheme of a search task processing device of this embodiment. It should be noted that the technical scheme of the search task processing device and the technical scheme of the search task processing method described above are based on the same concept. For details not described in detail in the technical scheme of the search task processing device, please refer to the description of the technical scheme of the search task processing method described above.

[0238] Corresponding to the above method embodiment, this specification also provides an embodiment of a training device for a target scoring model. Figure 8 FIG. 1 shows a schematic diagram of a training device for a target scoring model provided by an embodiment of this specification. Figure 8 As shown, the device includes:

[0239] An acquisition module 802 is configured to acquire a training sample set for a target search task, wherein the training sample set includes at least one sample node and a corresponding label score, and the label score is determined based on a correctness reward or an execution efficiency reward of the sample node;

[0240] The prediction module 804 is configured to obtain corresponding prediction scores based on the sample nodes in the training sample set through the initial scoring model;

[0241] The training module 806 is configured to determine the target loss of the initial scoring model based on the label score and predicted score of the sample node, train the initial scoring model based on the target loss, and obtain a trained target scoring model, wherein the target scoring model is used to score the target node in the above-mentioned search task processing method.

[0242] Optionally, the training module 806 is further configured to:

[0243] Determine the recall loss of the sample node based on the label score and prediction score of the sample node;

[0244] Based on the recall loss and the set threshold penalty factor, the target loss of the initial scoring model is determined.

[0245] Optionally, the sample node is a sample answer; the acquisition module 802 is further configured to:

[0246] Obtain test examples under the target search task;

[0247] Generate at least one sample answer corresponding to each sample question under the target search task, test each sample answer based on the test sample, and obtain test information of each test sample and each sample answer;

[0248] Determine a label score for each sample answer based on a correctness reward or an execution efficiency reward for each sample answer, wherein the correctness reward or the execution efficiency reward for the sample answer is determined based on test information of the sample answer;

[0249] Based on the answers to each sample and the corresponding label scores, a training sample set for the target search task is obtained.

[0250] Optionally, the acquisition module 802 is further configured to:

[0251] Obtain at least one sample question under the target search task and the reference answer to each sample question;

[0252] Generate at least one set of test examples corresponding to each sample problem;

[0253] The test samples are verified based on the reference answers to the sample questions to obtain the test samples that pass the verification.

[0254] Optionally, the acquisition module 802 is further configured to:

[0255] determining, based on the test information, whether a first sample answer has a test example that fails the test, wherein the first sample answer is any one of the sample answers;

[0256] If so, determine the correctness reward of the first sample answer based on the test information, and map the correctness reward of the first sample answer to a first threshold range to obtain a label score for the first sample answer;

[0257] If not, the execution efficiency reward of the first sample answer is determined based on the test information, and the execution efficiency reward of the first sample answer is mapped to the second threshold range to obtain the label score of the first sample answer.

[0258] Optionally, the acquisition module 802 is further configured to:

[0259] Obtaining a test result of whether the first sample answer passes the test on each test sample;

[0260] Determining a sample score for each test sample, wherein the sample score for the test sample is determined based on the answer scores of the passed sample answers, the total score of each sample answer, and the test result;

[0261] The correctness reward of the first sample answer is determined based on the sample scores of the test samples passed by the first sample answer, the total scores of the test samples, and the test results.

[0262] Optionally, the execution efficiency reward includes a time reward and / or a content reward; the acquisition module 802 is further configured to:

[0263] Based on the test information, determining the maximum execution time and the minimum execution time of the first sample answer on each test sample, and determining a time bonus score for the first sample answer based on the maximum execution time and the minimum execution time;

[0264] Based on the test information, determine the maximum content length and the minimum content length of each sample answer, and determine the content reward score of the first sample answer;

[0265] Based on the time reward score and / or the content reward score and in combination with a threshold value of the second mapping range, a label score of the first sample answer is determined.

[0266] Optionally, the device further includes an obtaining module configured to:

[0267] Obtain candidate generative models;

[0268] The language model layer in the candidate generation model is replaced with a linear layer and a mapping layer to obtain an initial scoring model.

[0269] In the embodiments of this specification, by designing a scientific reward formula and a dedicated data set, designing a specific loss formula, and training an initial scoring model, the reward model can adaptively adjust the search depth based on a set threshold, achieving the goal of quickly answering simple questions and deeply optimizing complex problems, thereby improving reasoning efficiency and accuracy.

[0270] Figure 9 A structural block diagram of a computing device provided by an embodiment of this specification is shown.

[0271] The computing device 900 includes:

[0272] Memory 910 and processor 920;

[0273] The memory 910 is used to store computer programs / instructions, and the processor 920 is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor 920, the steps of the above-mentioned search task processing method or target scoring model training method are implemented.

[0274] In one or more embodiments of this specification, the computing device 900 can be understood as an integrated intelligent terminal, including but not limited to a server, a desktop computer, a PC (Personal Computer), an all-in-one model machine, a mobile phone, a tablet computer or other portable intelligent terminal, etc., and the computing device can be pre-installed with the model in the above embodiments of this application.

[0275] Specifically, the computing device 900 can pre-install multiple types of models, including but not limited to models in the fields of natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thereby providing a diverse selection of models. In different product forms, the computing device 900 can support one or more model usage methods, including but not limited to model training, model calling, model fine-tuning, model deployment, model reasoning, and application. In some product forms, the computing device 900 also supports model management, including but not limited to multi-type model management (supporting the management of multiple types of models such as discriminants and generative models), model version control (supporting the control of different model versions), and model evaluation (using model evaluation tools to evaluate model performance and effectiveness). In other product forms, the computing device 900 can also create applications based on models and provide API (Application Programming Interface) calling capabilities. Models can be called into created applications through the API interface, and application management tools are also provided to enable management and monitoring of applications.

[0276] Furthermore, the computing device 900 may also include data management (supporting the creation and management of model tuning data sets), a training center (providing rich training resources to help users learn artificial intelligence technology), and basic management and control capabilities (providing enterprise-level basic management and control capabilities to ensure the security and efficient operation of the system). Through the above functions, a comprehensive, integrated artificial intelligence development, training, deployment and application device is provided.

[0277] Figure 10 A structural block diagram of an electronic device provided by an embodiment of this specification is shown.

[0278] The memory 1010 and the processor 1020 are connected via a bus 1030 ;

[0279] The memory 1010 is used to store computer programs / instructions, and the processor 1020 is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor 1020, the steps of the above-mentioned search task processing method or target scoring model training method are implemented.

[0280] Specifically, the components of the electronic device 1000 include but are not limited to a memory 1010 and a processor 1020. The processor 1020 and the memory 1010 may be connected via a bus 1030.

[0281] The electronic device 1000 may further include an access device 1040 that enables the electronic device 1000 to communicate with a database 1050 storing data via one or more networks 1060. Examples of these networks 1060 include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0282] In one embodiment of the present specification, the above components of the electronic device 1000 and Figure 10 Other components not shown in the figure may also be connected to each other, for example, via bus 1030. It should be understood that Figure 10 The electronic device structure block diagram shown is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0283] Electronic device 1000 may be any type of stationary or mobile electronic device, including a mobile computer or mobile electronic device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable electronic device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary electronic device such as a desktop computer or personal computer (PC). Electronic device 1000 may also be a mobile or stationary server.

[0284] The above is a schematic scheme of an electronic device of this embodiment. It should be noted that the technical scheme of the electronic device and the technical scheme of the above-mentioned search task processing method or target scoring model training method are based on the same concept. For details not described in detail in the technical scheme of the electronic device, please refer to the description of the technical scheme of the above-mentioned search task processing method or target scoring model training method.

[0285] An embodiment of the present specification also provides a computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned search task processing method or target scoring model training method.

[0286] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solution of the search task processing method or the target scoring model training method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the search task processing method or the target scoring model training method described above.

[0287] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned search task processing method or target scoring model training method.

[0288] The above is a schematic diagram of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product is based on the same concept as the technical solution of the aforementioned search task processing method or target scoring model training method. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the aforementioned search task processing method or target scoring model training method.

[0289] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0290] Computer programs / instructions include computer program code, which may be in source code, object code, executable files, or some intermediate form. Computer-readable media may include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. It should be noted that the content of computer-readable media may be appropriately expanded or reduced based on the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunications signals.

[0291] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0292] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0293] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A search task processing method, comprising: Determining a target node of a target search task, wherein the target search task is a task of updating and optimizing target generation content generated by a target generation model, the target generation model is a generative large model, the target search task includes a code search optimization task or a text search optimization task, and the target node is an answer that needs to be iteratively optimized; The scoring result of the target node is determined by a target scoring model, wherein the target scoring model is obtained based on at least one sample node and corresponding label scoring training, the sample node is a sample answer, and if there is a test sample that fails the test for the first sample answer, then the correctness reward of the first sample answer is determined based on the test information, and the correctness reward of the first sample answer is mapped to a first threshold range to obtain a label score for the first sample answer; if there is no test sample that fails the test for the first sample answer, then the execution efficiency reward of the first sample answer is determined based on the test information, and the execution efficiency reward of the first sample answer is mapped to a second threshold range to obtain a label score for the first sample answer, the first sample answer is any one of the sample answers, the correctness reward is used to evaluate whether the content of the sample node is correct, and the execution efficiency reward is used to evaluate the processing efficiency of the sample node; Node expansion is performed based on the scoring result of the target node to determine the search result of the target search task.

2. The method according to claim 1, wherein the node expansion is performed based on the scoring result of the target node to determine the search result of the target search task, comprising: Determining whether the scoring result of the target node satisfies a stop search constraint; If the stop search constraint is satisfied, the target node is used as the search result of the target search task; If the stop search constraint is not met, the target node is expanded to obtain an extended node, and the search results of the target search task are determined based on the current nodes of the target search task, wherein the current nodes of the target search task include the target node and the extended node.

3. The method according to claim 2, wherein the target node is target generated content; and the step of expanding the target node to obtain an expanded node comprises: Based on the set evaluation prompt words, the target generation model is used to generate the extended content corresponding to the target generation content, and the extended content is used as the extended node.

4. The method according to claim 2, wherein determining the search results of the target search task based on the current nodes of the target search task comprises: Determine the comprehensive scores of each of the nodes respectively, determine the node to be expanded from the nodes based on the comprehensive scores, use the node to be expanded as the updated target node, and return to the step of determining the scoring result of the target node through the target scoring model until the search result of the target search task is obtained.

5. The method according to claim 2, wherein determining whether the scoring result of the target node satisfies a stop search constraint comprises: When the score result of the target node is greater than a set score threshold, determining that the stop search constraint is satisfied; When the scoring result of the target node is less than or equal to the set scoring threshold, it is determined that the stop search constraint is not satisfied.

6. The method according to claim 1, before determining the target node of the target search task, further comprising: For the target generation task of the target scenario, obtaining the target generation content corresponding to the target generation task through the target generation model; A corresponding target search task is created based on the target generation content, wherein the target search task is used to update the target generation content.

7. A method for training a target scoring model, comprising: Obtain a training sample set under a target search task, wherein the target search task is a task of updating and optimizing the target generation content generated by a target generation model, the target generation model is a generative large model, the target search task includes a code search optimization task or a text search optimization task, the training sample set includes at least one sample node and a corresponding label score, the sample node is a sample answer, and if a first sample answer has a test sample that fails the test, then a correctness reward of the first sample answer is determined based on the test information, and the correctness reward of the first sample answer is mapped to a first threshold range to obtain a label score of the first sample answer; if the first sample answer does not have a test sample that fails the test, then an execution efficiency reward of the first sample answer is determined based on the test information, and the execution efficiency reward of the first sample answer is mapped to a second threshold range to obtain a label score of the first sample answer, the first sample answer is any one of the sample answers, the correctness reward is used to evaluate whether the content of the sample node is correct, and the execution efficiency reward is used to evaluate the processing efficiency of the sample node; Based on the sample nodes in the training sample set, obtaining corresponding prediction scores through an initial scoring model; Based on the label score of the sample node and the predicted score, the target loss of the initial scoring model is determined, and the initial scoring model is trained based on the target loss to obtain a trained target scoring model, wherein the target scoring model is used to score the target node in the search task processing method described in any one of claims 1 to 6 above.

8. The method according to claim 7, wherein determining the target loss of the initial scoring model based on the label score of the sample node and the predicted score comprises: Determining the recall loss of the sample node based on the label score of the sample node and the prediction score; Based on the recall loss and a set threshold penalty factor, a target loss of the initial scoring model is determined.

9. The method according to claim 7, wherein the sample node is a sample answer; and obtaining a training sample set for a target search task comprises: Obtaining test examples for the target search task; Generate at least one sample answer corresponding to each sample question under the target search task, test each sample answer based on the test sample, and obtain test information of each test sample and each sample answer; Determining a label score for each sample answer based on a correctness reward or an execution efficiency reward for each sample answer, wherein the correctness reward or the execution efficiency reward for the sample answer is determined based on test information of the sample answer; Based on the sample answers and corresponding label scores, a training sample set for the target search task is obtained.

10. The method according to claim 9, wherein obtaining test examples for the target search task comprises: Obtain at least one sample question under the target search task and a reference answer to each sample question; Generating at least one set of test examples corresponding to each of the sample problems; The test examples are verified based on the reference answers to the sample questions to obtain test examples that pass the verification.

11. The method according to claim 7, wherein determining the correctness reward of the first sample answer based on the test information comprises: Obtaining a test result of whether the first sample answer passes the test on each test example; Determining a sample score for each test sample, wherein the sample score for the test sample is determined based on the answer scores of the passed sample answers, the total score of each sample answer, and the test result; A correctness reward for the first sample answer is determined based on the sample scores of the test samples passed by the first sample answer, the total scores of the test samples, and the test result.

12. The method according to claim 7, wherein the execution efficiency reward includes a time reward and / or a content reward; determining the execution efficiency reward of the first sample answer based on the test information, mapping the execution efficiency reward of the first sample answer to a second threshold range, and obtaining a label score for the first sample answer comprises: Determining, based on the test information, a maximum execution time and a minimum execution time of the first sample answer on each test case, and determining a time bonus score for the first sample answer based on the maximum execution time and the minimum execution time; Based on the test information, determining the maximum content length and the minimum content length of each sample answer, and determining a content reward score for the first sample answer; Based on the time reward score and / or the content reward score, combined with a threshold value in the second threshold range, a label score for the first sample answer is determined.

13. The method according to claim 7, further comprising: Obtain candidate generative models; The language model layer in the candidate generation model is replaced with a linear layer and a mapping layer to obtain the initial scoring model.

14. A computing device comprising: memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the method according to any one of claims 1 to 13 are implemented.

15. An electronic device comprising: a memory and a processor, wherein the memory and the processor are connected via a bus; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the method according to any one of claims 1 to 13 are implemented.

16. A computer-readable storage medium storing a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the steps of the method according to any one of claims 1 to 13.

17. A computer program product comprising a computer program / instruction, which implements the steps of the method according to any one of claims 1 to 13 when executed by a processor.

Images