Method for filtering model multi-path generation results and related device
By combining the training sample set and the dynamic sampling quantity, and using a multi-dimensional evaluation mechanism to screen the multi-path generation results of the model, the problem of screening accuracy and efficiency in high-difficulty tasks is solved, and high-precision identification of a few correct results and resource optimization are achieved.
Patent Information
- Application Number
- CN202611302222.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-26
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to accurately identify a small number of correct results in multi-path result filtering under challenging tasks, and improper allocation of computational resources makes it difficult to balance filtering accuracy and efficiency.
By acquiring a pre-trained selection model, utilizing samples from the training sample set containing different distributions of the number of correct paths, dynamically adjusting the sampling quantity, and combining a multi-dimensional evaluation mechanism, the optimal candidate path is determined from multiple candidate paths.
It achieves high-precision screening of a small number of correct results under highly challenging tasks, dynamically balances computing resources, improves screening accuracy, and optimizes computing efficiency.
Smart Images

Figure CN123115381A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of artificial intelligence and natural language processing technology, and in particular to a method and related apparatus for screening results generated by a multi-path model. Background Technology
[0002] With the rapid development of large language model technology, improving model output quality through multi-path generation has become a mainstream technique. In practical applications, a test-time expansion strategy is adopted, which involves having the model generate multiple candidate results and then selecting the optimal solution to present to the user.
[0003] Existing screening methods mainly include mechanisms based on generation probability, general semantic matching, model self-evaluation, and voting consensus. However, these existing technologies have significant shortcomings when facing complex tasks or highly difficult problems. On the one hand, the training data of existing screening models are based on general corpora or manually constructed standard answers, which differ significantly from the distribution of real answers generated by the model during the actual reasoning stage, making it difficult for the screener to accurately assess the model's true generation ability. On the other hand, in highly difficult task scenarios, correct reasoning paths belong to the minority, and screening mechanisms based on majority voting or general confidence tend to select the majority results that conform to statistical laws but are substantially wrong, causing a minority of correct results to be submerged. In addition, generation strategies with a fixed number of paths cause computational redundancy when facing simple tasks, while when facing extremely difficult tasks, they are prone to insufficient sampling and failure to cover correct solutions, making it difficult to balance screening accuracy and computational efficiency.
[0004] Therefore, there is an urgent need for a screening method that can adapt to the actual generation distribution of the model, accurately identify the few correct results, and dynamically balance computing resources. Summary of the Invention
[0005] In view of the problems of existing screening methods, such as the mismatch between training data and real inference distribution, and the difficulty in identifying a few correct results in high-difficulty scenarios, this application provides a screening method and related apparatus for multi-path generation results of a model, so as to achieve high-precision screening of multi-path generation results and adaptive balance of computing resources. The specific solution is as follows:
[0006] The first aspect of this application provides a method for filtering the results generated by a multi-path model, including:
[0007] Obtain a pre-trained selection model, which is trained based on a training sample set containing samples with different distributions of the number of correct paths;
[0008] The number of samples is determined based on the input task;
[0009] Based on the stated number of samples, multiple candidate paths are generated using a generative model;
[0010] The input task and the multiple candidate paths are input into the selection model, the selection model determines the optimal candidate path from the multiple candidate paths, and outputs the identifier of the optimal candidate path.
[0011] In one possible implementation, the training sample set is constructed in the following way:
[0012] During the reinforcement learning phase, based on the task difficulty labels of each question in the question bank, the generative model generates multiple independent reasoning paths and complete answers for each question in parallel. Each question and its corresponding multiple reasoning paths and complete answers are used as a training sample to obtain a training sample set distributed hierarchically according to task difficulty, wherein the proportion of samples with difficulty levels higher than the preset level is not less than the preset proportion.
[0013] For each training sample, the multiple reasoning paths are comprehensively evaluated from multiple dimensions using expert system prompts, and the number of the optimal reasoning path and the reason for its selection are output as the training label for the training sample set.
[0014] In one possible implementation, the selection model is trained in the following manner:
[0015] The training sample set is associated with the corresponding training labels to construct supervised training data;
[0016] Based on the supervised training data, a multimodal large language model is trained under supervision. The input of the multimodal large language model is the questions in the training samples and their corresponding multiple inference paths. The output is the number of the optimal inference path. During the training process, the cross-entropy loss function is used to calculate the loss between the predicted number and the actual optimal number in the training labels.
[0017] The multimodal large language model after training is used as the selected model.
[0018] In one possible implementation, determining the number of samples based on the input task includes:
[0019] The difficulty level of the input task is assessed to obtain a difficulty rating.
[0020] The number of samples corresponding to the difficulty level is determined based on a preset mapping relationship between difficulty and number of samples.
[0021] In one possible implementation, the difficulty levels include a first difficulty level, a second difficulty level, a third difficulty level, and a fourth difficulty level, with the difficulty increasing sequentially from the first difficulty level to the fourth difficulty level.
[0022] The number of samples increases as the difficulty level increases.
[0023] In one possible implementation, the selection model is configured to perform multi-dimensional evaluation on the multiple candidate paths, the multi-dimensional evaluation including at least two of the following: answer correctness evaluation, reasoning step completeness evaluation, answer format standardization evaluation, and knowledge point expression accuracy evaluation.
[0024] The step of inputting the input task and the multiple candidate paths into the selection model includes:
[0025] The evaluation rule configuration information corresponding to the multi-dimensional evaluation is placed in the system prompt module, and the input task and the multiple candidate paths are placed in the user prompt module to construct prompt word data;
[0026] Input the prompt word data into the selection model.
[0027] In one possible implementation, the method further includes:
[0028] Based on the identifier of the optimal candidate path, the corresponding complete content is extracted from the multiple candidate paths as the final output result.
[0029] A second aspect of this application provides a device for filtering the results generated by a multi-path model, comprising:
[0030] The acquisition unit is used to acquire a pre-trained selection model, which is trained based on a training sample set containing samples with different distributions of the number of correct paths.
[0031] A sampling quantity determination unit is used to determine the sampling quantity based on the input task;
[0032] The candidate path generation unit is used to generate multiple candidate paths according to the number of samples using a generation model;
[0033] The filtering unit is used to input the input task and the multiple candidate paths into the selection model, determine the optimal candidate path from the multiple candidate paths through the selection model, and output the identifier of the optimal candidate path.
[0034] A third aspect of this application provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement a method for filtering multi-path generation results of a model according to the first aspect or any implementation thereof.
[0035] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:
[0036] The memory is used to store computer programs;
[0037] The processor is used to execute the computer program so that the electronic device can implement the method for filtering the multi-path generation results of the model in the first aspect or any implementation thereof.
[0038] The fifth aspect of this application provides a computer-readable storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the method for filtering model multipath generation results as described in the first aspect or any implementation thereof.
[0039] By employing the aforementioned technical solution, the method and related apparatus for filtering multi-path generation results provided in this application obtain a selection model trained on a training sample set containing different distributions of the number of correct paths. This allows the learning objective of the selection model to be directly aligned with the ability to identify the optimal solution from samples with differentiated accuracy distributions. Since the training sample set covers different distributions of the number of correct paths, especially including difficult samples with fewer correct paths, the selection model can learn feature representations that distinguish between a few correct paths and a majority of incorrect paths during training. This overcomes the deficiency of traditional filtering methods that fail in high-difficulty scenarios due to a minority of correct results, thus improving filtering accuracy. Furthermore, this method dynamically determines the number of samples based on the input task, rather than using a fixed number of generated paths. This mechanism allows the system to adaptively allocate computational resources according to the actual complexity of the task, ensuring sufficient sampling space to cover correct solutions for high-difficulty tasks while avoiding wasted computational power for low-difficulty tasks, achieving a dynamic balance between filtering accuracy and computational efficiency. Attached Figure Description
[0040] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0041] Figure 1 A flowchart illustrating a method for filtering multi-path generation results of a model, provided in an embodiment of this application;
[0042] Figure 2 A schematic diagram of the structure of a filtering device for model multipath generation results provided in an embodiment of this application;
[0043] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0044] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.
[0045] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.
[0046] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0047] Before introducing the embodiments of this application, we will first introduce the multi-path generation result filtering method in the prior art, so as to understand the technical background of the technical solution of this application and the technical problem to be solved.
[0048] Currently, one of the commonly used metrics for evaluating the generation capabilities of large language models is the pass@k metric. This metric represents the probability that at least one of the k candidate results generated by the model satisfies the correctness requirement. As the value of k increases, the model's pass@k accuracy improves significantly; for example, pass@5 accuracy is much higher than pass@1 accuracy. This is because the generation process of large language models is random; a single generation may result in logical errors, missing steps, calculation mistakes, or non-standard expressions, while multiple generation sessions can cover more correctness. However, in practical applications, it is impossible to present all k candidate results to the user. On the one hand, presenting multiple results affects the user experience; on the other hand, multiple results can confuse the user's understanding. Therefore, how to automatically select the optimal result from the k candidate results to present to the user, so that the accuracy of the final output approaches or even reaches the level of pass@k, has become a key technical problem to be solved.
[0049] In the existing technology, there are four main technical solutions for filtering multi-path generated results.
[0050] The first category is the selection method based on generation probability. This method utilizes the token-level probability distribution output during the autoregressive generation process of a large language model to calculate the overall generation confidence of each candidate result, and selects the result with the highest confidence as the final output. The implementation involves recording the conditional probability of each token at each step in the generation process of each candidate result, calculating the joint generation probability of the complete sequence using the chain rule, converting it to logarithmic probability to avoid numerical underflow, introducing a length normalization factor to obtain a perplexity index, and finally selecting the result with the lowest perplexity. This method requires no additional models or labeled data, has high computational efficiency, and is widely integrated into the basic inference frameworks of various large language models, such as the bundle search algorithm in the Transformers framework.
[0051] The second category is screening methods based on general semantic matching. This method evaluates the quality of candidate results by calculating the semantic similarity between the candidate result and the reference text. The core assumption is that the closer the candidate result is to the semantics of the reference text, the higher its quality. The implementation involves preparing a standard answer or benchmark answer as the reference text, using a pre-trained semantic representation model, such as BERT, Sentence-BERT, E5, BGE, etc., to encode the question, reference text, and all candidate results into semantic embedding vectors, calculating the cosine similarity between the candidate result and the reference text, and selecting the result with the highest similarity. Some improved schemes introduce multi-dimensional semantic matching, obtaining a comprehensive score through weighted summation.
[0052] The third category is the screening method based on large-scale model self-evaluation. This method utilizes the understanding and judgment capabilities of the large language model itself to directly evaluate and rank the quality of multiple candidate results. The implementation involves designing structured evaluation prompts based on the task type and evaluation criteria, concatenating the original question and all candidate results, and inputting them into the large model. The large model then scores each candidate result or directly ranks it, selecting the result with the highest score or the first-ranked result. Depending on the evaluation method, this can be divided into three categories: single-result scoring, multi-result ranking, and pairwise comparison.
[0053] The fourth category is selection methods based on voting mechanisms. This method uses the consistency among multiple candidate results to judge their correctness, assuming that the answer supported by more candidate results is more likely to be correct. The most representative of these is the self-consistency method. The implementation involves setting the temperature parameter of the large model to 0.5 to 1.0, generating k different inference paths and corresponding answers, extracting the final core answer from each candidate result, statistically analyzing the frequency of all core answers, and selecting the core answer with the highest frequency as the final output. To address the issue that the self-consistency method can only extract the final answer and cannot retain the complete inference process, the industry has proposed improved schemes such as path-to-answer binding voting, weighted voting, and semantic clustering voting.
[0054] While the above four existing technical solutions have achieved the filtering of multi-path generation results to a certain extent, they still have the following technical defects in practical applications, especially in high-difficulty task scenarios.
[0055] The fundamental flaw of generation probability-based screening methods lies in equating the statistical generation probability of a language model with the factual correctness of the content. Incorrect reasoning steps or calculation results receive a higher generation probability because they conform to the statistical laws of language. For example, in mathematical proofs, a logically flawed but fluently expressed incorrect answer has a lower perplexity than a logically rigorous but slightly complex correct answer. Furthermore, this method completely fails to assess quality dimensions such as the completeness of solution steps and the standardization of answer format.
[0056] The filtering method based on general semantic matching relies on the quality of the reference text and cannot distinguish between semantically similar but logically different results. For numerous cases where the semantics are similar but the correctness is clearly different—for example, two mathematical answers with the same numerical value but one correct and one incorrect derivation—the semantic matching method cannot effectively differentiate them. Furthermore, this method completely fails for tasks without standard answers, such as open-ended or exploratory questions.
[0057] Screening methods based on large-model self-evaluation suffer from a mismatch between evaluation and generation capabilities. Evaluation consistency is poor; multiple evaluations of the same question and candidate result set yield significantly different rankings. Position bias is severe; the large model tends to assign higher scores to higher-ranking candidates, leading to systematic bias in the screening results. Evaluation capability lags behind generation capability; while the large model can generate correct results, it cannot accurately determine which result is optimal, a problem particularly pronounced in scenarios requiring fine-grained differentiation of quality differences.
[0058] Voting-based selection methods fail to identify the minority of correct answers. The core assumption of voting mechanisms is that the majority is correct, but this assumption is completely invalid in high-difficulty problem scenarios. When the model faces a difficult problem, only a minority, or even just one, of the generated results from multiple paths may be correct—that is, 1 / N or 2 / N of the N paths may be correct. In this case, the voting mechanism will incorrectly classify the majority of incorrect results as the optimal solution, resulting in a final output accuracy far below the theoretical level of pass@k. Furthermore, voting mechanisms can only evaluate the correctness of the final answer, completely failing to assess the quality of the reasoning process and the standardization of the answer.
[0059] In summary, current technologies fail to address the core challenge of multi-path result selection: how to accurately identify the optimal solution from a minority of correct results in challenging tasks, while simultaneously ensuring the correctness of the answer, the completeness of the reasoning process, and the standardization of the answer format. Existing technologies do not consider the actual distribution characteristics of the model's responses in their training data and evaluation logic. They rely either on general linguistic statistical patterns or manually designed evaluation criteria, failing to utilize real-world response data generated by the multimodal model during the post-training reinforcement learning phase, nor do they perform specialized optimization training for the high-value scenario of a minority of correct responses. Therefore, when correct results become the minority, the selection accuracy of all existing methods drops sharply, failing to fully leverage the pass@k accuracy advantage brought by multi-path generation.
[0060] To address the aforementioned problems, embodiments of this application provide a method for filtering model multi-path generation results. The method for filtering model multi-path generation results of this application embodiment will be described in detail below with reference to the accompanying drawings.
[0061] Example 1
[0062] Reference Figure 1 , Figure 1 This is a flowchart illustrating a method for filtering multi-path generation results provided in this application. This method is primarily applied in the inference stage, dynamically adjusting the number of generated paths and using a dedicated selection model to filter them, thereby addressing the problem of difficulty in identifying correct results under challenging tasks. Figure 1 As shown in the embodiment of this application, a method for filtering the multi-path generation results of a model may include steps S100 to S400, which are described in detail below.
[0063] Step S100: Obtain a pre-trained selection model. The selection model is trained based on a training sample set, which contains samples with different distributions of the number of correct paths.
[0064] In the execution of this embodiment, the selection model is loaded or invoked as a pre-defined functional component, and its core capabilities stem from the specific data distribution used during the training phase. The training sample set not only includes regular samples but also specifically covers difficult samples with a small number of correct paths or even only a few correct paths. This data distribution characteristic enables the selection model to learn the key features that distinguish between a few correct paths and a majority of incorrect paths during inference, thereby overcoming the defect of traditional screening methods that fail when faced with highly difficult tasks due to a minority of correct results.
[0065] It should be understood that although this step emphasizes the compositional properties of the training sample set, when actually implementing the screening method, it is only necessary to obtain the already trained model instances, without having to repeat the training process.
[0066] Step S200: Determine the number of samples based on the input task.
[0067] The number of samples is not a fixed, preset value, but is dynamically determined based on the actual characteristics of the currently received input task. For example, the system can first evaluate the complexity or difficulty level of the input task and then map it to obtain a matching number of samples. For simple tasks, a smaller number of samples can be determined to avoid wasting computing power; for complex or high-difficulty tasks, a larger number of samples can be determined to increase the probability of covering the correct solution.
[0068] This dynamic determination mechanism achieves an adaptive balance between computational resource consumption and the upper limit of generation accuracy, avoiding the problems of redundancy in easy problems or insufficient sampling of difficult problems caused by the fixed path number strategy in the existing technology.
[0069] Step S300: Generate multiple candidate paths using a generative model based on the number of samples.
[0070] After determining the number of samples for the current input task, the generative model is controlled to generate the corresponding number of candidate paths in parallel or serially.
[0071] To ensure the diversity of candidate paths, random parameters, such as temperature, are introduced during the generation process, resulting in differences in reasoning logic, expression, or final conclusion among each candidate path. These candidate paths constitute a candidate pool to be screened, providing a sufficient basis for comparison in subsequent model selection. For example, if the sampling quantity determined in step S200 is 8, this step will generate 8 independent candidate paths.
[0072] Step S400: Input the input task and multiple candidate paths into the selection model, determine the optimal candidate path from the multiple candidate paths through the selection model, and output the identifier of the optimal candidate path.
[0073] The input to the selection model includes both the original input task context and all candidate paths generated in step S300. This allows the selection model to comprehensively compare and evaluate each candidate path based on a full understanding of the task intent.
[0074] It is important to note that the output of the selection model is an identifier for the optimal candidate path. This identifier can be the candidate path's number, index, or other unique marker, rather than directly generating new text content or modifying and rewriting the original content. For example, if the third of the eight generated candidate paths is determined to be the optimal one, the selection model will directly output the number 3 or its corresponding ID label.
[0075] This output method clearly defines the selection model's function as a discriminator rather than a generator, ensuring that the final output is entirely derived from the actual generation results of the generator model, thus avoiding new errors introduced by secondary generation. Through this process, even in challenging scenarios where the correct path is in the minority, the selection model can accurately identify it using its discriminative ability learned from specific distribution data, thereby bringing the accuracy of the final selection result close to the theoretical pass@k level.
[0076] Example 2
[0077] In some embodiments, to enable the selection model to accurately identify a few correct paths, the training sample set is constructed as follows: During the reinforcement learning phase, based on the task difficulty labels of each question in the question bank, the generative model generates multiple independent reasoning paths and complete answers for each question in parallel. Each question and its corresponding multiple reasoning paths and complete answers are used as a training sample to obtain a training sample set distributed hierarchically according to task difficulty, wherein the proportion of samples with difficulty levels higher than a preset level is not less than a preset proportion. For the multiple reasoning paths in each training sample, multi-dimensional comprehensive evaluation is performed using expert system prompts, and the number of the optimal reasoning path and the reason for its selection are output as the training label of the training sample set.
[0078] The core of the above construction method lies in the authenticity of the data source and the targeted distribution. The training samples are not manually compiled or simply pieced together from general corpora, but are directly collected from real sampled data generated by the generative model during the reinforcement learning phase. This means that the error types, logical flaws, and expression habits in the training data are completely consistent with the generation distribution of the model during actual inference, thus avoiding the filter failure problem caused by the distribution deviation between training and inference data. Simultaneously, to address the scarcity of correct results in highly difficult scenarios, the sample set adopts a stratified construction strategy based on task difficulty. For example, questions can be divided into four levels: easy, medium, difficult, and extremely difficult, ensuring that highly difficult and extremely difficult samples account for no less than 40% of the total samples. This mandatory stratified distribution ensures that the selected model is fully exposed to difficult samples with few or even only a few correct paths during training, forcing it to learn how to capture weak correct signals amidst a large number of error interferences, rather than simply memorizing the statistical patterns of simple tasks.
[0079] In the label generation stage, expert system prompts replace traditional manual annotation or simple rule matching. These expert system prompts incorporate fine-grained evaluation rules conforming to specific domain standards, following a decision-making logic of first disproving and eliminating obviously incorrect options, and then selecting the optimal path according to the standards. For example, when evaluating mathematical reasoning paths, the expert system not only verifies the numerical correctness of the final answer but also progressively checks the rationality of auxiliary line construction, the accuracy of formula references, and the completeness of proof steps. Only when a path meets the criteria in all key dimensions is it marked as the optimal reasoning path, and its number and specific selection reason are recorded. This high-quality supervision signal ensures that the learning objective of the selected model is aligned with the domain's gold standard, rather than simply being the result of majority voting.
[0080] Furthermore, the selection model is trained in the following way: the training sample set is associated with the corresponding training labels to construct supervised training data; based on the supervised training data, the multimodal large language model is trained under supervision, wherein the input of the multimodal large language model is the questions in the training samples and their corresponding multiple reasoning paths, and the output is the number of the optimal reasoning path. During the training process, the cross-entropy loss function is used to calculate the loss between the predicted number and the actual optimal number in the training labels; the multimodal large language model after training is used as the selection model.
[0081] This supervised training process models the selected task as either a classification or ranking problem. The model's input context includes the original question information and all candidate reasoning paths to be evaluated, enabling the model to perform comparative analysis within a global perspective. The model's output is strictly limited to the number of the optimal reasoning path, rather than generating new text or scores; this design reinforces the model's function as a discriminator.
[0082] For training optimization, the cross-entropy loss function is used as the objective function, which effectively measures the difference between the probability distribution predicted by the model and the true label distribution. Since the training samples contain a large number of difficult samples where the correct paths are few, the cross-entropy loss forces the model to invest more gradient update weights on these samples, thereby learning the key feature representations that distinguish between the few correct paths and the majority of incorrect paths. Compared to using mean squared error or other regression losses, cross-entropy loss has better convergence when handling discrete class selection tasks and is more adaptable to scenarios with class imbalance.
[0083] Through this targeted supervised training, the selection model ultimately acquires the ability to accurately identify the optimal solution from multiple candidate paths during testing, even if the optimal solution is outnumbered. This training method can be applied to the training scenario of the Best-of-N (BON) selection model, where N corresponds to the aforementioned sampling number. The goal of the BON selection model is to select the optimal solution from N candidate paths.
[0084] Example 3
[0085] In some embodiments, the specific implementation of determining the number of samples based on the input task includes: assessing the difficulty of the input task to obtain a difficulty level; and determining the number of samples corresponding to the difficulty level according to a preset mapping relationship between difficulty and number of samples.
[0086] This step concretizes the abstract dynamic determination process into two sub-stages: evaluation and mapping. In the difficulty evaluation stage, the system does not rely on manual pre-labeling but employs automated evaluation methods. As one feasible implementation, a lightweight difficulty classification model can be trained, taking statistical features or textual semantic features of historical tasks as input. These statistical features include average solution time, historical pass rate, etc., and directly output a discrete difficulty level. Another feasible implementation is to utilize the zero-shot reasoning capability of large language models, guiding the model to analyze the complexity of the current input task and return a difficulty label through preset prompts. This hierarchical design ensures that the solution is not limited to a specific evaluation algorithm. Whether based on traditional machine learning classifiers or semantic understanding based on generative models, as long as it can produce a level label representing the difficulty of the task, it falls within the scope of this application.
[0087] Furthermore, the difficulty levels include Level 1, Level 2, Level 3, and Level 4, with the difficulty increasing sequentially from Level 1 to Level 4; the number of samples increases with the increase of the difficulty level.
[0088] This positive correlation constitutes the core control logic of this embodiment. Taking one application scenario as an example, the difficulty level can be divided into four levels: the first difficulty level corresponds to simple tasks with a difficulty coefficient of 0 to 0.2, the second difficulty level corresponds to medium tasks with a difficulty coefficient of 0.2 to 0.6, the third difficulty level corresponds to difficult tasks with a difficulty coefficient of 0.6 to 0.8, and the fourth difficulty level corresponds to extremely difficult tasks with a difficulty coefficient of 0.8 to 1.0. It should be understood that the above division of difficulty coefficient ranges is only an illustrative example and can be adjusted according to the distribution of business data in actual deployment. For example, percentile ranking or cluster analysis can be used to dynamically define the boundaries of each level.
[0089] After determining the difficulty level, the system determines the corresponding number of samples based on a preset mapping table or mapping function. For example, for a simple task at the first difficulty level, the number of samples can be set to 3; for a medium task at the second difficulty level, the number of samples can be set to 5; for a difficult task at the third difficulty level, the number of samples can be set to 8; and for an extremely difficult task at the fourth difficulty level, the number of samples can be increased to 10 or even more. It should be noted that the above values are not fixed limitations; their core purpose is to reflect the trend of "the higher the difficulty, the more samples." In some embodiments, this mapping relationship can also be a continuous nonlinear function, rather than a discrete lookup table operation, as long as the number of samples and the difficulty level are monotonically increasing.
[0090] This embodiment achieves an adaptive balance between computational resources and generation accuracy through the aforementioned mechanism. From a technical perspective, for simple tasks, the generation model itself has a high single-generation accuracy, and a small number of candidate paths are sufficient to cover the correct solution. In this case, reducing the number of samples can significantly reduce inference latency and computational consumption, avoiding unnecessary redundancy. Conversely, for highly difficult tasks, the distribution of correct paths in the generation space is extremely sparse, belonging to a minority or even isolated points. Only by increasing the sampling scale can the probability of generating correct solutions be increased, thus providing "superior" models for subsequent selection. If a fixed small number of samples is still used in highly difficult scenarios, it is very easy for all candidate paths to become incorrect results, rendering the selection process meaningless. Therefore, this strategy of dynamically adjusting the number of samples according to difficulty essentially maximizes the overall system's operating efficiency while ensuring that the output accuracy of highly difficult tasks approaches the theoretical level of pass@k.
[0091] Example 4
[0092] In some embodiments, the model selection is configured to perform multi-dimensional evaluation on multiple candidate paths. The multi-dimensional evaluation includes at least two of the following: answer correctness evaluation, reasoning step completeness evaluation, answer format standardization evaluation, and knowledge point expression accuracy evaluation.
[0093] This feature concretizes the abstract screening capability into a set of configurable and composable evaluation dimensions. Answer correctness evaluation verifies whether the final conclusion or key intermediate result of the candidate path conforms to the facts or standard answer; reasoning step integrity evaluation checks for jumps, omissions, or circular arguments in the logical deduction chain; answer format conformity evaluation judges whether the output content conforms to the preset structural requirements or domain norms; and knowledge point expression accuracy evaluation focuses on the precision of professional terminology, concept definitions, and principle explanations.
[0094] In practical applications, the system does not necessarily require the use of all four dimensions every time. Instead, it flexibly selects at least two dimensions to combine based on the type of input task. For example, for mathematical calculation tasks, the system can prioritize combining the correctness assessment of the answer with the completeness assessment of the reasoning steps to ensure the accuracy of the numerical results and the rigor of the derivation process. For humanities essay or code generation tasks, the system can combine the standardization assessment of the answer format with the accuracy assessment of the knowledge point expression to ensure the structure of the output and the appropriateness of the use of professional terminology. This modular dimension combination mechanism allows the model selection to adapt to diverse task requirements, avoiding the adaptability bias of a single evaluation indicator in different scenarios.
[0095] Furthermore, the process of inputting the input task and multiple candidate paths into the selection model includes: placing the evaluation rule configuration information corresponding to the multi-dimensional evaluation into the system prompt module, placing the input task and multiple candidate paths into the user prompt module, constructing prompt word data; and inputting the prompt word data into the selection model.
[0096] The construction of this prompt word data employs a strict structured separation strategy. The system prompt module carries relatively static evaluation rule configuration information, including but not limited to the definitions of each evaluation dimension, scoring criteria, adjudication logic, and domain-specific canonical dictionaries. This content constitutes the "cognitive baseline" for selecting the model and remains unchanged throughout a single evaluation session. The user prompt module, on the other hand, carries the dynamically changing input task context and multiple candidate paths to be evaluated.
[0097] By physically isolating the evaluation rules from the content to be evaluated in terms of prompt word structure, this embodiment achieves multiple technical effects.
[0098] First, this separation effectively alleviates the positional bias problem commonly found in large language models. In traditional concatenation methods, candidate paths appearing before the prompt words are more likely to achieve high scores. However, by placing the rule at the beginning and embedding it in the system prompt module, the model can perform equal comparisons based on a unified benchmark when processing multiple candidate paths in the user prompt module, rather than being limited by the order in which the text appears.
[0099] Secondly, the structured configuration ensures consistency of the evaluation criteria between the training and inference phases. Since the evaluation rule configuration information in the system prompt module is consistent with, or even completely identical to, the expert system prompts used during training, the selection model can accurately reproduce the discrimination logic learned during training during inference, avoiding performance fluctuations caused by differences in prompt style. Finally, this design improves the system's maintainability and scalability. When it is necessary to adjust the evaluation criteria or adapt to new domains, only the rule configuration in the system prompt module needs to be updated, without retraining the model or modifying the underlying code, greatly reducing iteration costs.
[0100] It should be understood that although this embodiment uses the system prompt module and the user prompt module as examples, other functionally equivalent structured input forms can also be used in other implementations. As long as the logical decoupling between the evaluation rules and the content to be evaluated can be achieved, they are all within the protection scope of this application.
[0101] Example 5
[0102] In some embodiments, the filtering method further includes: extracting the corresponding complete content from multiple candidate paths as the final output result based on the identifier of the optimal candidate path.
[0103] This step constitutes the final execution loop of the screening process, realizing the transformation from selection decision to actual content delivery. After the selection model outputs the identifier of the optimal candidate path, the system uses this identifier as a retrieval key to backtrack to the previously generated and cached dataset of multiple candidate paths, accurately locating and retrieving the original data of the candidate path that matches the identifier. For example, if the identifier output by the selection model is the number "3", the system directly reads the complete text sequence at index 3 from the memory buffer or temporary storage area that stores multiple candidate paths.
[0104] The complete content emphasized here refers to the original generated content actually produced by the generative model during the inference stage, without any subsequent editing, polishing, summarizing, or rewriting. This extraction mechanism is fundamentally different from traditional generative evaluation. In existing technologies, some evaluation models, after determining that a certain result is superior, will attempt to combine the advantages of multiple results to regenerate an answer, or rewrite and optimize the selected result. However, in application scenarios with extremely high accuracy requirements, this secondary generation is prone to introducing new illusion errors or logical loopholes, causing the final output to deviate from the originally determined correct inference path. This embodiment, through rigorous extraction operations, ensures that the content finally presented to the user is completely consistent with the object evaluated by the selection model at the data level, thereby completely avoiding the uncertainty risks introduced by the post-processing stage. At the same time, this feature also clarifies that the selection model only undertakes the function of discrimination and selection, and does not undertake the function of content generation. This not only reduces the computational overhead during inference, but also makes the system's error attribution clearer and more traceable.
[0105] It should be understood that although the above example uses a numerical identifier as the identifier, in other implementations, the identifier can also be a unique string ID, a hash value, or a memory pointer, etc. As long as a one-to-one mapping relationship between the identifier and the original candidate path can be established and lossless extraction is supported, it is within the protection scope of this application.
[0106] Example 6
[0107] To more clearly illustrate the effect of the technical solution of this application in practical applications, the following detailed explanation will be based on a practical application in the context of intelligent education Q&A.
[0108] In this scenario, suppose a user inputs a high school geometry problem via a terminal device. This problem involves complex solid geometry proofs and calculations, representing a typical high-difficulty task in a K-12 education setting. Upon receiving this input, the system first assesses its difficulty, classifying it as level four, or extremely difficult. Based on a pre-defined mapping between difficulty and sample size, the system dynamically determines the current sample size to be 8.
[0109] The generative model generates eight independent candidate paths in parallel, based on a predetermined sampling size of eight. Due to the extremely high difficulty of the problem, the probability of the generative model producing a correct solution in a single inference is low, resulting in a typical minority correct distribution characteristic among the eight candidate paths. For example, among these eight candidate paths, only the third path provides the correct auxiliary line construction and a rigorous proof derivation, while the other seven paths all have varying degrees of logical flaws, incorrect auxiliary lines, or calculation errors. This 1 / N correct path ratio accurately reflects the true distribution of multi-path generation results in high-difficulty task scenarios and is a core pain point that existing technologies struggle to effectively address.
[0110] Subsequently, the system inputs the original high school geometry problem and eight generated candidate paths into a pre-trained selection model. Based on its training-stage assessment capabilities specific to educational scenarios, the selection model performs a multi-dimensional comprehensive evaluation of these eight candidate paths. In this example, the evaluation dimensions include at least two or more combinations of the following: answer correctness assessment, reasoning step completeness assessment, answer format standardization assessment, and knowledge point expression accuracy assessment. The selection model not only verifies the correctness of the final calculation result but also thoroughly examines the rationality of the auxiliary line construction, the accuracy of theorem citations, and the completeness of the proof chain in each path. After comprehensive comparison, the selection model accurately identifies the third candidate path as the optimal candidate path, despite being in the minority in number, because it meets expert-level grading standards across all key evaluation dimensions. The model outputs a corresponding identifier, such as the number 3.
[0111] In contrast, if the existing majority voting mechanism is used for selection, since 7 out of 8 paths point to the same erroneous conclusion or use similar flawed approaches, the voting mechanism will incorrectly determine the majority of erroneous paths as the optimal solution based on frequency statistics, resulting in a completely wrong final output. The selection model in this embodiment, however, effectively overcomes this inherent flaw of the majority voting method by leveraging its discriminative ability trained on a training sample set containing different distributions of correct paths. It can accurately pinpoint the optimal solution even when a few correct results are overwhelmed by a large number of incorrect results.
[0112] Finally, based on the identifier 3 output by the selected model, the system extracts the complete content of the third path from the eight cached candidate paths and presents it to the user as the final output. This output includes complete solution steps, a standardized format, and accurate descriptions of knowledge points, fully preserving the high-quality content originally generated by the generative model without any secondary editing or rewriting.
[0113] Through the above process, this application has successfully achieved the goal of selecting a few correct solutions from multiple candidate paths in the application scenario of intelligent education Q&A, which has extremely high accuracy requirements. This makes the final output accuracy close to or even reach the theoretical level of pass@8, which is significantly better than traditional fixed sampling or majority voting methods. This fully verifies the effectiveness and technological progress of the technical solution in high-difficulty task scenarios.
[0114] It should be understood that the above description using high school mathematics geometry problems as an example is merely illustrative. The screening method of this application is also applicable to other intelligent application scenarios that require multi-dimensional quality assessment, such as Chinese composition correction, science experiment step analysis, code generation and debugging. As long as the input task has the requirement of multi-path generation and there are a few correct solutions, the technical solution provided in this application can be used to solve the identification problem.
[0115] The above describes a method for filtering model multipath generation results provided by embodiments of this application. The following describes the apparatus for implementing the above-described method for filtering model multipath generation results.
[0116] Please see Figure 2 , Figure 2 This is a schematic diagram of a device for filtering the results of a model multipath generation, provided in an embodiment of this application. Figure 2 As shown, the filtering device for the multi-path generation results of this model includes:
[0117] The acquisition unit 11 is used to acquire a pre-trained selection model, which is trained based on a training sample set containing samples with different distributions of the number of correct paths.
[0118] The sampling quantity determination unit 12 is used to determine the sampling quantity based on the input task;
[0119] The candidate path generation unit 13 is used to generate multiple candidate paths according to the sampling quantity using a generation model;
[0120] The filtering unit 14 is used to input the input task and the multiple candidate paths into the selection model, determine the optimal candidate path from the multiple candidate paths through the selection model, and output the identifier of the optimal candidate path.
[0121] Each unit in the multi-path generation result filtering device of the above model can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in a computer device, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each unit.
[0122] This application also provides an electronic device in its embodiments. (See reference...) Figure 3 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0123] like Figure 3 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0124] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0125] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement a method for filtering the multi-path generation results of any model provided in this application.
[0126] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the multi-path generation result filtering methods provided in this application.
[0127] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0129] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.
[0130] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
Claims
1. A method of filtering model multipath generated results, the method comprising: include: Obtain a pre-trained selection model, which is trained based on a training sample set containing samples with different distributions of the number of correct paths; The number of samples is determined based on the input task; Based on the stated number of samples, multiple candidate paths are generated using a generative model; The input task and the multiple candidate paths are input into the selection model, the selection model determines the optimal candidate path from the multiple candidate paths, and outputs the identifier of the optimal candidate path.
2. The method according to claim 1, characterized in that, The training sample set is constructed in the following way: During the reinforcement learning phase, based on the task difficulty labels of each question in the question bank, the generative model generates multiple independent reasoning paths and complete answers for each question in parallel. Each question and its corresponding multiple reasoning paths and complete answers are used as a training sample to obtain a training sample set distributed hierarchically according to task difficulty, wherein the proportion of samples with difficulty levels higher than the preset level is not less than the preset proportion. For each training sample, the multiple reasoning paths are comprehensively evaluated from multiple dimensions using expert system prompts, and the number of the optimal reasoning path and the reason for its selection are output as the training label for the training sample set.
3. The method according to claim 2, characterized in that, The selection model was trained in the following way: The training sample set is associated with the corresponding training labels to construct supervised training data; Based on the supervised training data, a multimodal large language model is trained under supervision. The input of the multimodal large language model is the questions in the training samples and their corresponding multiple inference paths. The output is the number of the optimal inference path. During the training process, the cross-entropy loss function is used to calculate the loss between the predicted number and the actual optimal number in the training labels. The multimodal large language model after training is used as the selected model.
4. The method according to claim 1, characterized in that, The process of determining the number of samples based on the input task includes: The difficulty level of the input task is assessed to obtain a difficulty rating. The number of samples corresponding to the difficulty level is determined based on a preset mapping relationship between difficulty and number of samples.
5. The method according to claim 4, characterized in that, The difficulty levels include a first difficulty level, a second difficulty level, a third difficulty level, and a fourth difficulty level, with the difficulty increasing sequentially from the first difficulty level to the fourth difficulty level. The number of samples increases as the difficulty level increases.
6. The method according to claim 1, characterized in that, The selection model is configured to perform multi-dimensional evaluation on the multiple candidate paths. The multi-dimensional evaluation includes at least two of the following: answer correctness evaluation, reasoning step completeness evaluation, answer format standardization evaluation, and knowledge point expression accuracy evaluation. The step of inputting the input task and the multiple candidate paths into the selection model includes: The evaluation rule configuration information corresponding to the multi-dimensional evaluation is placed in the system prompt module, and the input task and the multiple candidate paths are placed in the user prompt module to construct prompt word data; Input the prompt word data into the selection model.
7. The method according to claim 1, characterized in that, The method further includes: Based on the identifier of the optimal candidate path, the corresponding complete content is extracted from the multiple candidate paths as the final output result.
8. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the method for filtering model multipath generation results as described in any one of claims 1 to 7.
9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the method for filtering model multipath generation results as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The storage medium carries one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the method for filtering model multipath generation results as described in any one of claims 1 to 7.
