Method and device for training model, storage medium and electronic equipment
By employing a two-stage training framework and injecting difficulty and redundancy hypnotic labels, the "overthinking" problem in large-scale reasoning models is addressed, enabling autonomous difficulty recognition and redundancy elimination, thereby optimizing reasoning efficiency and reducing computational costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-10
AI Technical Summary
Large-scale reasoning models suffer from the "overthinking" problem, which leads to unnecessary computational costs and increased latency, and lacks the ability to autonomously recognize difficulty.
Through a two-stage training framework, difficulty cognition and redundancy elimination training are carried out separately. Difficulty hypnosis labels and redundancy elimination hypnosis labels are injected to enable the model to autonomously recognize difficulty and redundancy, and guide the model to autonomously select reasoning strategies.
While maintaining inference accuracy, the length of task inference is reduced, significantly lowering computational costs and optimizing inference efficiency.
Smart Images
Figure CN121834347A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to artificial intelligence technology, and more particularly to a method, apparatus, storage medium, and electronic device for training models. Background Technology
[0002] Current large-scale inference models commonly suffer from the "overthinking" problem, generating lengthy and complex inference chains even when faced with simple tasks. This leads to unnecessary increases in computational costs and latency, the root cause of which is the model's lack of autonomous difficulty perception capabilities like humans. With the widespread deployment of large-scale inference models in practical applications, inference efficiency directly impacts service costs and user experience, necessitating urgent optimization of computational efficiency while maintaining performance. Summary of the Invention
[0003] The purpose of the embodiments in this specification is to provide a method, apparatus, storage medium, and electronic device for training models.
[0004] This specification provides a method for training a model, proposing an efficiency optimization method for reasoning thought chains based on difficulty and redundancy cognition. This method addresses the "overthinking" problem prevalent in current large-scale reasoning models, achieving an autonomous difficulty cognition mechanism. By enabling the model to assess task difficulty similar to humans, it adaptively selects reasoning strategies based on the difficulty of the problem. Through guided text label training, external instructions are internalized into autonomous model behavior, achieving intelligent reasoning mode switching. A two-stage training framework progressively trains the model on difficulty cognition and redundancy elimination, gradually endowing it with these abilities, ensuring stable and controllable training results. This method reduces task reasoning length and significantly lowers computational costs while maintaining reasoning accuracy. The method includes: By injecting a difficulty hypnotic label corresponding to the target problem, the target model is trained to recognize the difficulty of the target problem. The difficulty hypnotic label is used to guide the target model to use the reasoning strategy corresponding to the difficulty information of the target problem. Redundant steps in the reasoning process of the target model regarding the target problem are identified, and redundancy elimination hypnotic labels are injected into the redundant steps to enable the target model to perform redundant cognitive training for the target problem. The redundancy elimination hypnotic labels are used to guide the target model to identify and eliminate the redundant steps.
[0005] Furthermore, the method also includes: Obtain the difficulty information corresponding to the target problem; The difficulty hypnosis tag corresponding to the target question is determined based on the difficulty information.
[0006] Furthermore, obtaining the difficulty information corresponding to the target problem includes: The target problem is input into the trained first model to obtain the difficulty information output by the first model.
[0007] Furthermore, obtaining the difficulty information corresponding to the target problem includes: By analyzing the knowledge content involved in the target problem, the difficulty information corresponding to the target problem is obtained.
[0008] Furthermore, the difficulty hypnotic tag corresponding to the injected target question includes: Inject the difficulty hypnosis tag corresponding to the target question into the prompt information corresponding to the target question.
[0009] Furthermore, the reasoning strategy includes a rapid intuition strategy or a slow, deliberate strategy.
[0010] Furthermore, the step of identifying redundant steps in the reasoning process of the target model regarding the target problem includes: If the reasoning strategy includes a slow and deliberate strategy, redundant steps in the reasoning process of the target model regarding the target problem are identified, and redundant elimination hypnotic labels are injected into the redundant steps, so that the target model can perform redundant cognitive training for the target problem.
[0011] Furthermore, the redundant steps include at least one of invalid reflection steps, invalid loop steps, invalid repetition steps, and invalid reasoning steps.
[0012] Furthermore, the step of identifying redundant steps in the reasoning process of the target model regarding the target problem includes: Obtain the first thought chain of the target model in the reasoning process regarding the target problem; Identify redundant steps in the first thought chain.
[0013] Furthermore, the step of identifying redundant steps in the first thought chain includes: The first thought chain and its corresponding prompts are input into a trained large model to obtain redundant steps in the first thought chain output by the large model.
[0014] Furthermore, the step of injecting redundancy removal hypnotic tags for the redundant steps includes: Based on the redundancy type of the redundant steps, determine the redundancy elimination hypnosis tag corresponding to the redundant steps; The redundant steps are addressed by injecting the redundant elimination hypnotic tag.
[0015] Furthermore, the step of injecting the redundancy elimination hypnotic tag for the redundant step includes: The redundancy elimination hypnosis tag is injected into the prompt information corresponding to the target problem for the redundant steps.
[0016] Furthermore, injecting the redundancy elimination hypnotic tag into the prompt information corresponding to the target question for the redundant steps includes: Obtain the second thought chain of the target model in the reasoning process regarding the target problem; Obtain at least one step in the second thought chain that precedes the redundant step; The at least one step and the redundancy elimination hypnosis tag are injected into the prompt information corresponding to the target problem.
[0017] Furthermore, the step of injecting the redundancy elimination hypnotic tag for the redundant step includes: During the reasoning process of the target model, redundant steps are intercepted, and redundant elimination hypnotic tags are injected.
[0018] This specification also provides an embodiment of a method for model reasoning, including: The target problem is input into the target model, wherein the target model is trained based on the method for training the model described in the embodiments of this specification; The target model identifies the difficulty information of the target problem, uses the reasoning strategy corresponding to the difficulty information to reason about the target problem, identifies and eliminates redundant steps in the reasoning process, and obtains the answer to the target problem output by the target model.
[0019] This specification also provides an apparatus for training a model, comprising: The difficulty cognition training module is used to enable the target model to perform difficulty cognition training for the target problem by injecting a difficulty hypnotic label corresponding to the target problem. The difficulty hypnotic label is used to guide the target model to use the reasoning strategy corresponding to the difficulty information of the target problem. A redundant cognitive training module is used to identify redundant steps in the reasoning process of the target model regarding the target problem. By injecting redundancy elimination hypnotic labels into the redundant steps, the target model is trained to perform redundant cognitive training on the target problem. The redundancy elimination hypnotic labels are used to guide the target model to identify and eliminate the redundant steps.
[0020] This specification also provides an apparatus for model reasoning, comprising: An input module is used to input the target problem into the target model, wherein the target model has completed difficulty cognitive training and redundancy cognitive training; The output module is used to enable the target model to identify the difficulty information of the target problem, use the reasoning strategy corresponding to the difficulty information to reason about the target problem, identify and eliminate redundant steps in the reasoning process, and obtain the answer corresponding to the target problem output by the target model.
[0021] This specification also provides a storage medium storing a computer program adapted to be loaded by a processor and to execute the steps of the method described above.
[0022] This specification also provides an electronic device, including a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method described above.
[0023] This specification also provides a computer program product that stores at least one instruction, characterized in that the at least one instruction, when executed by a processor, implements the steps of the above-described method.
[0024] Based on the embodiments of this specification, a method for optimizing the efficiency of reasoning thought chains based on difficulty and redundancy cognition is proposed. This method solves the "overthinking" problem commonly found in current large-scale reasoning models, realizes the model's autonomous difficulty cognition mechanism, and enables the model to have a task difficulty assessment ability similar to humans. It adaptively selects reasoning strategies according to the difficulty of the problem. Through guided text label training, external instructions are internalized into the model's autonomous behavior, realizing intelligent reasoning mode switching. Through a two-stage training framework, progressive training is carried out on difficulty cognition and redundancy elimination respectively, gradually endowing the model with difficulty cognition and redundancy cognition capabilities, ensuring stable and controllable training results. It can reduce the task reasoning length while maintaining reasoning accuracy, and significantly reduce computational costs. Attached Figure Description
[0025] Figure 1 A flowchart illustrating a method for training a model provided in an embodiment of this specification; Figure 2 A flowchart illustrating a method for training a model provided in an embodiment of this specification; Figure 3 A flowchart illustrating a method for training a model provided in an embodiment of this specification; Figure 4 A flowchart illustrating a method for training a model provided in an embodiment of this specification; Figure 5 A schematic diagram of a device for training a model provided in an embodiment of this specification; Figure 6A schematic diagram of a device for model reasoning provided in an embodiment of this specification; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation
[0026] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.
[0027] Please see Figure 1 This is a flowchart illustrating a method for training a model provided in an embodiment of this specification. In this embodiment, the method for training a model is applied to a device for training a model (hereinafter referred to as a "model training device") or an electronic device equipped with a model training device. The following will focus on... Figure 1 The process shown will be described in detail. The method for training the model may specifically include the following steps: S102, by injecting a difficulty hypnotic label corresponding to the target problem, the target model is trained to recognize the difficulty of the target problem, wherein the difficulty hypnotic label is used to guide the target model to use the reasoning strategy corresponding to the difficulty information of the target problem.
[0028] In some embodiments, the target model is a large reasoning model (LRM), which is a large language model specifically designed for complex reasoning tasks and has multi-step logical reasoning capabilities.
[0029] In some embodiments, the difficulty hypnosis label is used to guide the target model to use the reasoning strategy corresponding to the difficulty information of the target problem to reason about the target problem. The difficulty information is used to characterize the difficulty level of the target problem. The difficulty information can be in numerical form. For example, the larger the value, the more complex the target problem is, and the smaller the value, the simpler the target problem is. Alternatively, the difficulty information can also be in string form (dividing the difficulty information into multiple levels), such as the strings "simple" and "complex". This example embodiment does not make any special limitation on the specific form of the difficulty information.
[0030] In some embodiments, an inference strategy refers to a technical system that optimizes intermediate computation processes through specific methods before the target model generates the final answer to the target problem during the inference phase. Its core objective is to improve inference efficiency and accuracy. In some embodiments, inference strategies include, but are not limited to, rapid intuition and slow deliberation. For example, the difficulty information "simple" corresponds to the inference strategy "rapid intuition," and the difficulty information "complex" corresponds to the inference strategy "slow deliberation." This example embodiment does not specifically limit the content of the inference strategy.
[0031] In some embodiments, the target problem is derived from a constructed heterogeneous dataset, which includes both simple tasks (e.g., GSM8K (Grade School Math 8K)) and difficult tasks (e.g., MATH (Mathematics Competition Dataset) and AIME (American Invitational Mathematics Examination)).
[0032] In some embodiments, a difficulty hypnosis label is a guiding text label. This label is injected into the target model before or during reasoning for the target problem, enabling the model to internalize different reasoning patterns. For example, a difficulty hypnosis label could be "This is a simple question, let's think quickly," corresponding to the difficulty information "simple." Another example is "This requires careful analysis, let's think step by step," corresponding to the difficulty information "complex." This example embodiment does not specifically limit the content of the difficulty hypnosis label. The solution in this specification innovatively uses the concept of "hypnosis," internalizing reasoning patterns through guiding text, transforming external instructions into the model's internal behavior, achieving self-guided reasoning, and introducing the concept of psychological hypnosis into the model training methodology.
[0033] In some embodiments, by injecting a difficulty hypnosis label into the target model before or during reasoning for the target problem, the target model is trained at the problem level based on the difficulty hypnosis label (i.e., difficulty cognition training). Through training, the target model establishes a mapping relationship between difficulty information and reasoning strategies, learns to autonomously identify the difficulty information of the problem, and learns to autonomously select the reasoning strategy mapped by the difficulty information to reason about the problem.
[0034] S104, Identify redundant steps in the reasoning process of the target model regarding the target problem, and inject redundancy elimination hypnotic labels into the redundant steps to enable the target model to perform redundant cognitive training for the target problem, wherein the redundancy elimination hypnotic labels are used to guide the target model to identify and eliminate the redundant steps.
[0035] In some embodiments, redundant steps include, but are not limited to, ineffective reflection steps (e.g., excessive self-criticism, constantly overturning previous reasonable reasoning, and being unable to move forward), ineffective loop steps (e.g., getting stuck in a meaningless loop that does not substantially contribute to solving the problem), ineffective repetitive steps (e.g., repeatedly stating the same content), and ineffective reasoning steps (e.g., easily generating the correct answer, but still reasoning), etc., which are not specifically limited in this example embodiment.
[0036] In some embodiments, the redundancy elimination hypnosis label is used to guide the target model to identify and eliminate redundant steps in the reasoning process when reasoning about the target problem. In some embodiments, the redundancy elimination hypnosis label is a guiding text label. By injecting the guiding text label into the target model before or during reasoning about the target problem, the target model can identify and eliminate redundant steps in the reasoning process. For example, the redundancy elimination hypnosis label could be "Everything seems ready now, let's move on." This example embodiment does not specifically limit the content of the redundancy elimination hypnosis label.
[0037] In some embodiments, after the target model undergoes difficulty recognition training for the target problem, redundancy elimination hypnotic labels are injected into the target model before or during reasoning for the target problem. This allows the target model to undergo step-level training (i.e., redundancy recognition training) based on these labels, enabling it to learn to identify and eliminate redundant steps in the reasoning process, truncate unnecessary reasoning content, and optimize reasoning efficiency. The solution in this specification employs a two-stage progressive training framework (difficulty recognition training + redundancy recognition training), employing a divide-and-conquer training strategy. It first learns difficulty recognition, then learns redundancy elimination, avoiding the complexity of one-time training, ensuring stable improvement in capabilities, and achieving a systematic and progressive training scheme for reasoning efficiency.
[0038] According to the embodiments of this specification, a method for optimizing the efficiency of reasoning thought chains based on difficulty and redundancy cognition is proposed. This method solves the "overthinking" problem that is common in current large-scale reasoning models, and realizes the model's autonomous difficulty cognition mechanism. By enabling the model to have a task difficulty assessment ability similar to humans, it adaptively selects reasoning strategies according to the difficulty of the problem. Through guided text label training, external instructions are internalized into the model's autonomous behavior, realizing intelligent reasoning mode switching. Through a two-stage training framework (difficulty cognition training + redundancy cognition training), progressive training is carried out on difficulty cognition and redundancy elimination respectively, gradually endowing the model with difficulty cognition and redundancy cognition capabilities, ensuring stable and controllable training results. It can reduce the task reasoning length while maintaining reasoning accuracy, and significantly reduce computational costs.
[0039] In some embodiments, the method further includes: obtaining difficulty information corresponding to the target problem; and determining a difficulty hypnotic label corresponding to the target problem based on the difficulty information. In some embodiments, it is necessary to first determine the difficulty information of the target problem, and then, according to a preset mapping relationship between difficulty information and difficulty hypnotic labels, use the difficulty hypnotic label mapped by the difficulty information as the difficulty hypnotic label of the target problem. For example, if the difficulty information is in numerical form, according to a preset mapping relationship between difficulty numerical ranges and difficulty hypnotic labels, the difficulty hypnotic label mapped to the difficulty numerical range into which the difficulty information falls is used as the difficulty hypnotic label of the target problem. In some embodiments, the target problem can also be input into a trained model (distinct from the target model), and the difficulty hypnotic label output by the model can be used as the difficulty hypnotic label of the target problem.
[0040] In some embodiments, obtaining the difficulty information corresponding to the target problem includes: inputting the target problem into a trained first model and obtaining the difficulty information output by the first model. In some embodiments, the target problem can be input into a trained first model (distinct from the target model), and the difficulty information output by the first model can be used as the difficulty information corresponding to the target problem. In this example embodiment, the model structure and model parameters of the first model are not specifically limited.
[0041] In some embodiments, obtaining the difficulty information corresponding to the target problem includes: analyzing the knowledge content involved in the target problem to obtain the difficulty information corresponding to the target problem. In some embodiments, the difficulty information of the target problem is determined based on the number and relevance of knowledge points in the analysis results, or based on the level of the cognitive process of the knowledge content in the analysis results (including but not limited to memory / recognition, understanding, application, analysis, synthesis / creation, evaluation, etc.), or based on the contextual complexity and / or contextual novelty of the knowledge content in the analysis results.
[0042] In some embodiments, injecting the difficulty hypnosis tag corresponding to the target question includes: injecting the difficulty hypnosis tag corresponding to the target question into the prompt information corresponding to the target question. In some embodiments, the corresponding difficulty hypnosis tag can be injected into the prompt information (prompt) corresponding to the target question, that is, the target question and the prompt containing the difficulty hypnosis tag corresponding to the target question are input into the target model, for example, the difficulty hypnosis tag is added to the end of the original prompt of the target question.
[0043] In some embodiments, the reasoning strategy includes a fast intuition strategy or a slow deliberate strategy. In some embodiments, the fast intuition strategy and the slow deliberate strategy are derived from human thinking patterns in cognitive psychology. By applying cognitive science thinking theories to model reasoning optimization, the target model can employ different reasoning strategies to reason about problems of varying difficulty, similar to how humans use different reasoning strategies for problems of different difficulties.
[0044] In some embodiments, identifying redundant steps in the reasoning process of the target model regarding the target problem, and injecting redundancy elimination hypnotic labels into the redundant steps to enable the target model to perform redundant cognitive training for the target problem, includes: if the reasoning strategy includes a slow and deliberate strategy, identifying redundant steps in the reasoning process of the target model regarding the target problem, and injecting redundancy elimination hypnotic labels into the redundant steps to enable the target model to perform redundant cognitive training for the target problem. In some embodiments, after the target model performs difficulty cognitive training for the target problem, redundant cognitive training based on redundancy elimination hypnotic labels will only be performed if the reasoning strategy corresponding to the difficulty information of the target problem is a slow and deliberate strategy. For example, redundant cognitive training based on redundancy elimination hypnotic labels will only be performed if the target problem corresponds to the difficulty information "complex". In some embodiments, if the reasoning strategy corresponding to the difficulty information of the target problem is a fast and intuitive strategy, the target model will not continue to perform redundant cognitive training for the target problem. For example, if the target problem corresponds to the difficulty information "simple", the target model will not continue to perform redundant cognitive training for the target problem.
[0045] In some embodiments, the redundant steps include at least one of invalid reflection steps, invalid looping steps, invalid repetition steps, and invalid reasoning steps. In some embodiments, redundant steps include, but are not limited to, invalid reflection steps (e.g., excessive self-criticism, constantly overturning previous reasonable reasoning, and being unable to move forward), invalid looping steps (e.g., getting stuck in a meaningless loop that does not substantially contribute to solving the problem), invalid repetition steps (e.g., repeatedly stating the same content), invalid reasoning steps (e.g., easily generating the correct answer, but still reasoning), etc., and this example embodiment does not specifically limit them.
[0046] In some embodiments, identifying redundant steps in the reasoning process of the target model regarding the target question includes: obtaining a first thought chain in the reasoning process of the target model regarding the target question; and identifying redundant steps in the first thought chain. In some embodiments, the target question is input into the target model, and after the target model outputs the answer corresponding to the target question, the first thought chain in the reasoning process of the target model regarding the target question is obtained, and then redundant steps in the first thought chain are identified. Here, the thought chain refers to the process by which the target model displays its internal reasoning steps before giving a final answer to the target question. For example, redundant steps can be identified from the first thought chain through semantic analysis; alternatively, redundant steps can be identified from the first thought chain through keyword analysis according to preset rules; or, the first thought chain can be input into a trained second model (different from the target model) to obtain the redundant steps output by the second model. In this example embodiment, the model structure and model parameters of the second model are not specifically limited.
[0047] In some embodiments, injecting redundancy elimination hypnosis tags for the redundant steps includes: determining the redundancy elimination hypnosis tag corresponding to the redundant step based on the redundancy type of the redundant step; and injecting the redundancy elimination hypnosis tag into the redundant step. In some embodiments, it is necessary to first determine the redundancy type of the redundant step. Redundancy types include, but are not limited to, invalid reflection, invalid loops, invalid repeated steps, invalid reasoning, etc., and this example embodiment does not impose any special limitations on this. In some embodiments, based on the preset mapping relationship between redundancy types and redundancy elimination hypnosis tags, the redundancy elimination hypnosis tag mapped to the redundancy type of the redundant step is used as the redundancy elimination hypnosis tag corresponding to the redundant step. Then, the redundancy elimination hypnosis tag corresponding to the redundant step is injected into the target model for the redundant steps in the reasoning process of the target model regarding the target problem.
[0048] In some embodiments, injecting the redundancy elimination hypnosis tag for the redundant step includes: injecting the redundancy elimination hypnosis tag for the redundant step into the prompt information corresponding to the target question. In some embodiments, the corresponding redundancy elimination hypnosis tag can be injected into the prompt information (prompt) corresponding to the target question for the redundant step. For example, not only is identification information for identifying the redundant step injected into the prompt, but also the redundancy elimination hypnosis tag corresponding to the redundant step is injected into the prompt. Another example is that not only is the feature information of the redundant step injected into the prompt, but also the redundancy elimination hypnosis tag corresponding to the redundant step is injected into the prompt. Yet another example is that not only is the specific content of the redundant step directly injected into the prompt, but also the redundancy elimination hypnosis tag corresponding to the redundant step is injected into the prompt.
[0049] In some embodiments, injecting the redundancy elimination hypnosis tag into the prompt information corresponding to the target question for the redundant step includes: obtaining a second thought chain of the target model in the reasoning process about the target question; obtaining at least one step in the second thought chain that precedes the redundant step; and injecting the at least one step and the redundancy elimination hypnosis tag into the prompt information corresponding to the target question. In some embodiments, after the target model outputs the answer to the target question, the second thought chain of the target model in the reasoning process about the target question is obtained, and redundant steps in the first thought chain are identified. The specific identification method is the same as or similar to the method for identifying redundant steps in the second thought chain described above, and will not be repeated here. In some embodiments, at least one step that precedes the redundant step is obtained from the second thought chain, and then the at least one step and the redundancy elimination hypnosis tag corresponding to the redundant step are injected into the prompt information corresponding to the target question. Specifically, this may involve injecting identification information for identifying the at least one step, or injecting feature information of the at least one step, or injecting the specific content of the at least one step.
[0050] In some embodiments, injecting the redundancy elimination hypnosis tag into the redundant step includes: intercepting the redundant step during the inference process of the target model and injecting the redundancy elimination hypnosis tag. In some embodiments, during the inference process of the target model, without modifying the target model, the redundant step can be intercepted using a hook (a programming mechanism that allows you to insert and run your own custom code at a specific node in the execution of a system, framework, or program), and the redundancy elimination hypnosis tag corresponding to the redundant step can be injected during the inference process. This allows the target model to continue the remaining inference for the target problem based on the redundancy elimination hypnosis tag, building upon the previous step corresponding to the redundant step. Alternatively, the redundant step can be rolled back during the inference process (i.e., the redundant step is undone, restoring the inference process to the state before the execution of the redundant step), and after the rollback is completed, the target model can continue the remaining inference for the target problem based on the redundancy elimination hypnosis tag, building upon the previous step corresponding to the redundant step.
[0051] Figure 2This is a flowchart illustrating a method for model inference provided in an embodiment of this specification. In this embodiment, the method for model inference is applied to a model inference apparatus (hereinafter referred to as a "model inference apparatus") or an electronic device equipped with a model inference apparatus as described in this embodiment. The following will focus on... Figure 2 The process shown will be described in detail. The method for model inference may specifically include the following steps: S202, the target problem is input into the target model, wherein the target model is trained based on the method for training the model described in the embodiments of this specification. In some embodiments, the target problem, the target model, and the training method of the target model have been described in detail above and will not be repeated here.
[0052] S204, the target model identifies the difficulty information of the target problem, uses the reasoning strategy corresponding to the difficulty information to reason about the target problem, identifies and eliminates redundant steps in the reasoning process, and obtains the answer to the target problem output by the target model. In some embodiments, the target model autonomously identifies the difficulty information of the target problem, autonomously uses the reasoning strategy corresponding to the difficulty information to reason about the target problem, and autonomously identifies and eliminates redundant steps in the reasoning process. Then, it outputs the final answer to the target problem obtained after the reasoning is completed to the target model. This can reduce the reasoning length of the target problem, optimize reasoning efficiency, and significantly reduce computational costs while ensuring the reasoning accuracy of the target problem. The specific process has been detailed above and will not be repeated here.
[0053] Figure 3 This is a flowchart illustrating a method for training a model, as provided in an embodiment of this specification.
[0054] like Figure 3 As shown, the input questions (including simple and complex questions) are first obtained. In the first stage, the model is trained to recognize the difficulty through a difficulty hypnosis mechanism. Corresponding difficulty hypnosis labels are injected for simple and difficult tasks to guide different reasoning modes. In the second stage, the model is trained to recognize redundancy. Redundancy is detected and marked by a decision-maker. Redundancy is identified and optimized in the reasoning process, so that the optimized model has autonomous difficulty recognition ability.
[0055] Figure 4 This is a flowchart illustrating a method for training a model, as provided in an embodiment of this specification.
[0056] like Figure 4As shown, the input question is first obtained. Then, in the first stage, the model is trained to recognize the difficulty by injecting difficulty hypnosis labels. By judging the difficulty of the question, if it is a simple question, the corresponding reasoning strategy for simple questions (quick intuition) is adopted to obtain the optimized model output, which is the efficient reasoning result. If it is a complex question, the corresponding reasoning strategy for complex questions (slow and deliberate) is adopted. Then, the model enters the second stage, where it is trained to recognize redundancy. The decision-maker analyzes the reasoning process to complete the redundancy detection, identify and eliminate redundant steps in the reasoning process, and obtain the optimized model output, which is the efficient reasoning result.
[0057] Figure 5 This is a schematic diagram of a device for training a model, provided in an embodiment of this specification. This device (hereinafter referred to as "model training device 1") can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the model training device 1 includes a difficulty recognition training module 11 and a redundancy recognition training module 12.
[0058] The difficulty cognition training module 11 is used to enable the target model to perform difficulty cognition training for the target problem by injecting a difficulty hypnotic label corresponding to the target problem. The difficulty hypnotic label is used to guide the target model to use the reasoning strategy corresponding to the difficulty information of the target problem. The redundant cognitive training module 12 is used to identify redundant steps in the reasoning process of the target model regarding the target problem. By injecting redundancy elimination hypnotic labels into the redundant steps, the target model is trained to perform redundant cognitive training on the target problem. The redundancy elimination hypnotic labels are used to guide the target model to identify and eliminate the redundant steps.
[0059] In some embodiments, the model training device 1 is further configured to: obtain difficulty information corresponding to the target problem; and determine the difficulty hypnosis label corresponding to the target problem based on the difficulty information.
[0060] In some embodiments, obtaining the difficulty information corresponding to the target problem includes: inputting the target problem into a trained first model to obtain the difficulty information output by the first model.
[0061] In some embodiments, obtaining the difficulty information corresponding to the target problem includes: obtaining the difficulty information corresponding to the target problem by analyzing the knowledge content involved in the target problem.
[0062] In some embodiments, injecting the difficulty hypnotic tag corresponding to the target question includes: injecting the difficulty hypnotic tag corresponding to the target question into the prompt information corresponding to the target question.
[0063] In some embodiments, the reasoning strategy includes a fast intuition strategy or a slow deliberate strategy.
[0064] In some embodiments, identifying redundant steps in the reasoning process of the target model regarding the target problem, and injecting redundancy elimination hypnotic labels into the redundant steps to enable the target model to perform redundant cognitive training for the target problem, includes: if the reasoning strategy includes a slow and deliberate strategy, identifying redundant steps in the reasoning process of the target model regarding the target problem, and injecting redundancy elimination hypnotic labels into the redundant steps to enable the target model to perform redundant cognitive training for the target problem.
[0065] In some embodiments, the redundant steps include at least one of invalid reflection steps, invalid loop steps, invalid repetition steps, and invalid reasoning steps.
[0066] In some embodiments, identifying redundant steps in the reasoning process of the target model regarding the target problem includes: obtaining a first thought chain of the target model in the reasoning process of the target problem; and identifying redundant steps in the first thought chain.
[0067] In some embodiments, injecting a redundancy elimination hypnotic tag for the redundant step includes: determining a redundancy elimination hypnotic tag corresponding to the redundant step based on the redundancy type of the redundant step; and injecting the redundancy elimination hypnotic tag for the redundant step.
[0068] In some embodiments, injecting the redundancy elimination hypnosis tag into the redundant step includes: injecting the redundancy elimination hypnosis tag into the prompt information corresponding to the target question for the redundant step.
[0069] In some embodiments, injecting the redundancy elimination hypnotic tag into the prompt information corresponding to the target question for the redundant step includes: obtaining a second thought chain of the target model in the reasoning process about the target question; obtaining at least one step in the second thought chain that is located before the redundant step; and injecting the at least one step and the redundancy elimination hypnotic tag into the prompt information corresponding to the target question.
[0070] In some embodiments, injecting the redundancy elimination hypnosis tag for the redundant step includes: intercepting the redundant step during the inference process of the target model and injecting the redundancy elimination hypnosis tag.
[0071] The above-described apparatus embodiments correspond to the aforementioned method embodiments. For detailed descriptions, please refer to the description in the method embodiments section; further details will not be repeated here. The apparatus embodiments are derived from the corresponding method embodiments and have the same technical effects. For detailed descriptions, please refer to the corresponding method embodiments.
[0072] Figure 6 This is a schematic diagram of a device for model reasoning provided in an embodiment of this specification. This device (hereinafter referred to as "model reasoning device 2") can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the model reasoning device 2 includes an input module 21 and an output module 22.
[0073] Input module 21 is used to input the target problem into the target model, wherein the target model has completed difficulty cognitive training and redundancy cognitive training; The output module 22 is used to enable the target model to identify the difficulty information of the target problem, use the reasoning strategy corresponding to the difficulty information to reason about the target problem, identify and eliminate redundant steps in the reasoning process, and obtain the answer corresponding to the target problem output by the target model.
[0074] The above-described apparatus embodiments correspond to the aforementioned method embodiments. For detailed descriptions, please refer to the description in the method embodiments section; further details will not be repeated here. The apparatus embodiments are derived from the corresponding method embodiments and have the same technical effects. For detailed descriptions, please refer to the corresponding method embodiments.
[0075] This specification also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in this specification.
[0076] This specification also provides a computer program product that stores at least one instruction, which is loaded by the processor and executes the method described in this specification embodiment.
[0077] This specification also provides an electronic device, including a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and execute the method described in the embodiments of this specification.
[0078] The embodiments in this specification also provide Figure 7 The diagram shows the structure of the electronic device. Figure 7At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the above method.
[0079] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.
[0080] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0081] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0084] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0085] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0086] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0087] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.
Claims
1. A method for training a model, comprising: By injecting a difficulty hypnotic label corresponding to the target problem, the target model is trained to recognize the difficulty of the target problem. The difficulty hypnotic label is used to guide the target model to use the reasoning strategy corresponding to the difficulty information of the target problem. Redundant steps in the reasoning process of the target model regarding the target problem are identified, and redundancy elimination hypnotic labels are injected into the redundant steps to enable the target model to perform redundant cognitive training for the target problem. The redundancy elimination hypnotic labels are used to guide the target model to identify and eliminate the redundant steps.
2. The method according to claim 1, further comprising: Obtain the difficulty information corresponding to the target problem; The difficulty hypnosis tag corresponding to the target question is determined based on the difficulty information.
3. The method according to claim 2, wherein obtaining the difficulty information corresponding to the target problem includes: The target problem is input into the trained first model to obtain the difficulty information output by the first model.
4. According to the method described in claim 2, obtaining the difficulty information corresponding to the target problem includes: By analyzing the knowledge content involved in the target problem, the difficulty information corresponding to the target problem is obtained.
5. The method according to claim 1, wherein the injected target question corresponding to a difficulty hypnotic tag comprises: Inject the difficulty hypnosis tag corresponding to the target question into the prompt information corresponding to the target question.
6. The method according to claim 1, wherein the reasoning strategy includes a fast intuition strategy or a slow deliberate strategy.
7. The method according to claim 6, wherein identifying redundant steps in the reasoning process of the target model regarding the target problem, and injecting redundancy-eliminating hypnotic labels into the redundant steps to enable the target model to perform redundant cognitive training for the target problem, comprises: If the reasoning strategy includes a slow and deliberate strategy, redundant steps in the reasoning process of the target model regarding the target problem are identified, and redundant elimination hypnotic labels are injected into the redundant steps, so that the target model can perform redundant cognitive training for the target problem.
8. The method according to claim 1 or 7, wherein the redundant steps include at least one of invalid reflection steps, invalid loop steps, invalid repetition steps, and invalid reasoning steps.
9. The method according to claim 1 or 7, wherein the step of identifying redundant steps in the reasoning process of the target model regarding the target problem comprises: Obtain the first thought chain of the target model in the reasoning process regarding the target problem; Identify redundant steps in the first thought chain.
10. The method according to claim 1 or 7, wherein the step of injecting redundancy-removal hypnotic tags for the redundant step comprises: Based on the redundancy type of the redundant steps, determine the redundancy elimination hypnosis tag corresponding to the redundant steps; The redundant steps are addressed by injecting the redundant elimination hypnotic tag.
11. The method of claim 10, wherein injecting the redundancy-eliminating hypnotic tag for the redundant step comprises: The redundancy elimination hypnosis tag is injected into the prompt information corresponding to the target problem for the redundant steps.
12. The method according to claim 11, wherein injecting the redundancy elimination hypnotic tag into the prompt information corresponding to the target question for the redundant step includes: Obtain the second thought chain of the target model in the reasoning process regarding the target problem; Obtain at least one step in the second thought chain that precedes the redundant step; The at least one step and the redundancy elimination hypnosis tag are injected into the prompt information corresponding to the target problem.
13. The method of claim 12, wherein injecting the redundancy-eliminating hypnotic tag for the redundant step comprises: During the reasoning process of the target model, redundant steps are intercepted, and redundant elimination hypnotic tags are injected.
14. A method for model reasoning, comprising: The target problem is input into the target model, wherein the target model is trained based on the method described in any one of claims 1 to 13; The target model identifies the difficulty information of the target problem, uses the reasoning strategy corresponding to the difficulty information to reason about the target problem, identifies and eliminates redundant steps in the reasoning process, and obtains the answer to the target problem output by the target model.
15. An apparatus for training a model, comprising: The difficulty cognition training module is used to enable the target model to perform difficulty cognition training for the target problem by injecting a difficulty hypnotic label corresponding to the target problem. The difficulty hypnotic label is used to guide the target model to use the reasoning strategy corresponding to the difficulty information of the target problem. A redundant cognitive training module is used to identify redundant steps in the reasoning process of the target model regarding the target problem. By injecting redundancy elimination hypnotic labels into the redundant steps, the target model is trained to perform redundant cognitive training on the target problem. The redundancy elimination hypnotic labels are used to guide the target model to identify and eliminate the redundant steps.
16. An apparatus for model reasoning, comprising: An input module is used to input the target problem into the target model, wherein the target model has completed difficulty cognitive training and redundancy cognitive training; The output module is used to enable the target model to identify the difficulty information of the target problem, use the reasoning strategy corresponding to the difficulty information to reason about the target problem, identify and eliminate redundant steps in the reasoning process, and obtain the answer corresponding to the target problem output by the target model.
17. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 14.
18. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as claimed in any one of claims 1 to 14.
19. A computer program product having at least one instruction stored thereon, characterized in that, When the at least one instruction is executed by the processor, it implements the steps of the method according to any one of claims 1 to 14.