A scientific reasoning method, device, medium and product based on a scientific reasoning model

Abstract

This application discloses a scientific reasoning method, device, medium, and product based on a scientific reasoning model, relating to the general field of scientific reasoning. The method includes acquiring a scientific question to be reasoned about; and determining a reasoning result based on the scientific question using a scientific reasoning model. The scientific reasoning model is obtained by training and optimizing a reasoning trajectory generation model using a scientific reasoning benchmark dataset through an information gain-guided quality learning phase and a dynamically length-controlled efficiency learning phase. The scientific reasoning benchmark datasets include the MMLU-Pro dataset, the SuperGPQA dataset, and the AGI-Eval dataset. This application improves the quality and efficiency of scientific reasoning.

Description

Technical Field

[0001] This application relates to the field of general scientific reasoning, and in particular to a scientific reasoning method, device, medium and product based on a scientific reasoning model. Background Technology

[0002] As large-scale language models (LLMs) have evolved from knowledge retrieval systems to cognitive reasoning systems, their application in scientific reasoning has made groundbreaking progress. For example, in professional fields such as mathematical problem-solving, logical deduction, medical diagnosis, and mathematical proof, models like DeepSeek-R1 and GPT-5-Nano have demonstrated near-professional-level capabilities, becoming important support for promoting artificial intelligence towards Artificial General Intelligence (AGI). However, general scientific reasoning, as a core capability for cultivating AI scientists and assisting in natural science discovery, still faces unique and unresolved key challenges. Existing technological solutions have significant limitations in practical applications, making it difficult to meet the dual demands of high precision and high efficiency for scientific research and engineering implementation.

[0003] Current scientific reasoning models generally adopt a single-objective optimization paradigm, leading to a dilemma between quality and efficiency optimization: on the one hand, quality-first solutions utilize GRPO (Group Relative Policy Optimization) and DAPO (Decoupled Clipping and Dynamic Sampling Policy) to optimize quality. While reinforcement learning algorithms such as optimization (decoupling pruning and dynamic sampling strategy optimization) can improve the accuracy of inference paths and thus increase the correctness of answers to some extent, they can also lead to a surge in the length of inference sequences. For example, in complex tasks such as physical formula derivation and biological pathology analysis, the number of tokens (a token is a fragment of words used in natural language processing, which can be words, letters, numbers, punctuation marks, etc., and is the smallest unit of text processed by a natural language model) in the inference process often exceeds 1,000, significantly increasing computational overhead and making it difficult for the model to meet the practical deployment requirements of low latency and low cost. On the other hand, efficiency-first solutions control computational costs through mechanisms such as dynamic early stopping and fixed length constraints, but at the expense of the integrity of inference. In multi-step subject-specific inference, insufficient inference depth often leads to logical breaks, ultimately causing the accuracy to drop by more than 15%. This contradiction of increasing accuracy at the expense of cost and reducing cost at the expense of rigor has become a core obstacle to the practical application of scientific inference models.

[0004] Meanwhile, existing reinforcement learning-based reasoning optimization methods have significant flaws in reward design. They generally rely on the correctness of the final answer to design reward functions, neglecting the evaluation of the quality of the reasoning process itself, thus leading to trivial group problems. When all reasoning trajectories lead to the correct answer, the reward signal tends to be consistent, and the model cannot effectively distinguish between high-quality rigorous reasoning and low-quality redundant reasoning, causing the policy gradient contribution to fail and making it difficult to learn efficient reasoning patterns. More importantly, these methods lack the ability to quantify the information gain of the reasoning process, making it impossible to determine whether the reasoning steps truly reduce the uncertainty of the answer. They are prone to generating reasoning chains containing redundant padding, such as repeating known conditions or calling on knowledge from disciplines unrelated to the problem, which wastes computational resources and reduces the interpretability of the reasoning results.

[0005] Furthermore, most mainstream scientific reasoning models rely on domain-adaptive pre-training on large-scale scientific corpora. Models such as SciGLM-6B and Intern-S1-mini improve performance by fine-tuning on specific subject corpora. However, this approach has limitations in domain adaptability and generalization ability. On the one hand, knowledge injection alone cannot directly transform accumulated subject knowledge into reliable symbolic reasoning logic. Although the model can accurately call up formulas from subjects such as Newton's laws in physics and reaction equations in chemistry, logical fallacies frequently occur in multi-step derivation, such as confusing causal relationships and misusing the applicable conditions of formulas. On the other hand, training data is mostly limited to basic questions in subjects such as physics and chemistry. When faced with interdisciplinary reasoning scenarios (such as the analysis of disease diagnosis and treatment equipment at the intersection of biomedicine and engineering) or open-domain scientific problems (such as experimental design and scientific hypothesis verification), the generalization accuracy of the model will decrease, making it difficult to adapt to the complex and diverse needs of scientific research. Summary of the Invention

[0006] The purpose of this application is to provide a scientific reasoning method, device, medium, and product based on a scientific reasoning model to improve the accuracy of reasoning results.

[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a scientific reasoning method based on a scientific reasoning model, including: Obtain the scientific question to be reasoned; Based on the scientific question to be reasoned about, the reasoning result is determined using a scientific reasoning model; The scientific reasoning model is obtained by training and optimizing the reasoning trajectory generation model using a scientific reasoning benchmark dataset through a quality learning phase guided by information gain and an efficiency learning phase with dynamic length control. The scientific reasoning benchmark dataset includes the MMLU-Pro dataset, the SuperGPQA dataset, and the AGI-Eval dataset.

[0008] In one embodiment, the information gain-guided quality learning phase specifically includes: A set number of scientific questions are sampled from the scientific reasoning benchmark dataset using stratified sampling, and a pre-trained large language model is used to generate distillation samples containing multi-step reasoning trajectories and final answers for each scientific question, forming a distillation dataset; The backbone model was subjected to supervised fine-tuning using the distillation dataset to obtain a reference model; the backbone model was either the Qwen2.5-7B model or the LLaMA3.1-8B model. For the scientific question, the first confidence level of the reference model when directly outputting the answer and the second confidence level when outputting the answer after generating the inference trajectory are obtained respectively, and the difference between the second confidence level and the first confidence level is calculated as the information gain; Based on the aforementioned information gain, output format compliance, and answer correctness, a comprehensive reward function is constructed; Based on the comprehensive reward function and the reference model, the current inference trajectory generation model is updated using a group relative strategy optimization algorithm to obtain the updated inference trajectory generation model.

[0009] In one embodiment, the comprehensive reward function is: ; in, As a comprehensive reward; A reward will be given based on the format; if the format is compliant, then... If the format is not compliant, then ; A reward is given for accuracy; if the final answer matches the standard answer, then... If the final answer differs from the standard answer, then ; Rewards are given for information gain. , For coefficients, , For temperature parameters, .

[0010] In one embodiment, based on the comprehensive reward function and the reference model, the current inference trajectory generation model is updated using a group relative policy optimization algorithm to obtain an updated inference trajectory generation model, specifically including: For each scientific question, sample G different inference trajectories from the current inference trajectory generation model; Calculate the overall reward for each inference trajectory, the average reward and standard deviation of the rewards for G inference trajectories, and then calculate the standardized within-group advantage for the i-th inference trajectory; Based on the standardized intra-group advantage of the i-th inference trajectory, the deviation between the current inference trajectory generation model and the reference model is constrained by KL divergence regularization. The parameters of the current inference trajectory generation model are updated by the objective function of the group relative strategy optimization algorithm, resulting in the updated inference trajectory generation model.

[0011] In one embodiment, the objective function is: ; in, The objective function is... It is the expectation of scientific reasoning benchmark D; q represents the logarithmic probability of the inference trajectory generated by the current inference trajectory generation model, where q is the science question. For scientific question q A line of reasoning, For the first The final answer corresponding to each reasoning trajectory; Here is the KL divergence regularization coefficient; ; It is a reference model; This indicates that the KL divergence regularization constraint ensures that the current inference trajectory generation model does not deviate from the reference model.

[0012] In one embodiment, the efficiency learning phase of dynamic length control specifically includes: For each scientific question in the current batch, the correct reasoning trajectory is selected from several reasoning trajectories generated by the model, and the shortest reasoning length among them is used as the dynamic length threshold for the corresponding scientific question. Construct an efficiency reward function that includes an accuracy reward and a length penalty term; wherein the length penalty term is determined based on the current inference trajectory length and the dynamic length threshold; Based on the efficiency reward function, the updated reasoning trajectory generation model is updated using a group relative strategy optimization algorithm to obtain a scientific reasoning model.

[0013] In one embodiment, the efficiency reward function is: ; in, Reward for efficiency; This is the adjustment coefficient; The length of the currently generated inference trail; This is the dynamic length threshold.

[0014] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described scientific reasoning method based on a scientific reasoning model.

[0015] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned scientific reasoning method based on a scientific reasoning model.

[0016] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned scientific reasoning method based on a scientific reasoning model.

[0017] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a scientific reasoning method, device, medium, and product based on a scientific reasoning model, which acquires a scientific question to be reasoned about; and determines the reasoning result based on the scientific question using a scientific reasoning model. The scientific reasoning model is obtained by training and optimizing a reasoning trajectory generation model using a scientific reasoning benchmark dataset through a quality learning phase guided by information gain and an efficiency learning phase with dynamic length control. The scientific reasoning benchmark dataset includes the MMLU-Pro dataset, the SuperGPQA dataset, and the AGI-Eval dataset. This application achieves high-precision, low-overhead collaborative optimization of scientific reasoning through a hierarchical design that first establishes a quality foundation and then optimizes efficiency: In the first stage of quality learning, a reward mechanism is designed based on information gain, quantifying the mutual information between the reasoning process and the correctness of the answer to guide the model to learn rigorous and valuable reasoning patterns, ensuring that each step of reasoning effectively reduces the uncertainty of the answer; in the second stage of efficiency optimization, a dynamic length control strategy is introduced, using the shortest length of correct reasoning in the current task as an adaptive threshold to precisely prune redundant steps and avoid ineffective computation without sacrificing reasoning accuracy. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a scientific reasoning method based on a scientific reasoning model, provided as an embodiment of this application; Figure 2An overview diagram of the SynergiSci framework provided for an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a two-stage optimization method for synergistic quality and efficiency in scientific reasoning. It addresses the problem of the incompatibility between quality and efficiency in existing scientific reasoning models by employing an information gain quantification mechanism (comparing the logit difference between direct answers and answers with reasoning), two-stage hierarchical training (first building a high-quality reasoning foundation using SFT+GRPO, then dynamically thresholding for redundancy), adaptive length thresholding (adapting to problem complexity based on the shortest length of correct reasoning in the current batch), and multi-dimensional evaluation fusion (combining accuracy, reasoning length, and AES score). This method differs from single-objective or fixed-constraint schemes, can be applied to scenarios in mathematics, physics, and medicine, is compatible with mainstream open-source models, and demonstrates strong versatility and practical applicability.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] In one exemplary embodiment, such as Figure 1 and Figure 2 As shown, a scientific reasoning method based on a scientific reasoning model is provided, including the following steps: S1: Obtain the scientific question to be reasoned.

[0024] S2: Based on the scientific question to be reasoned, use a scientific reasoning model to determine the reasoning result.

[0025] The scientific reasoning model is obtained by training and optimizing the reasoning trajectory generation model using a scientific reasoning benchmark dataset through a quality learning phase guided by information gain and an efficiency learning phase with dynamic length control. The scientific reasoning benchmark dataset includes the MMLU-Pro dataset, the SuperGPQA dataset, and the AGI-Eval dataset.

[0026] In this embodiment, a collaborative scientific reasoning framework (SynergiSci) is proposed. This is a two-stage optimization framework that decomposes quality learning and efficiency optimization into two collaborative stages, providing a standardized solution for deploying high-quality and efficient scientific reasoning models.

[0027] First, prepare the data.

[0028] Some of the data sources, sampling methods, and processing procedures are based on publicly available benchmark datasets and standardized processing paradigms in the field of scientific reasoning, ensuring data quality and experimental reproducibility.

[0029] To verify the effectiveness of the SynergiSci framework in in-domain adaptation and out-of-domain generalization scenarios, a dataset architecture combining core training, in-domain validation, and out-of-domain testing was adopted. The dataset selection followed the principles of multi-disciplinary coverage, graded difficulty, and verifiability. The in-domain datasets used for core training and validation included MMLU-Pro, SuperGPQA, and AGI-Eval. These datasets cover scientific reasoning tasks from basic to graduate level, supporting multi-step reasoning ability and subject knowledge coverage assessment. To comprehensively examine the model's generalization ability, three out-of-domain test sets were specifically constructed, each containing 1000 samples: SciQ (scientific question answering dataset), ScienceQA (scientific multiple-choice dataset), and SciEval (scientific evaluation dataset), for evaluating generalization ability. These datasets were not used in the training process and can be used to verify the model's reasoning stability and adaptability in unfamiliar scenarios.

[0030] The final dataset system comprises two categories: a core dataset within the domain and a generalization test set outside the domain. The core dataset within the domain originates from three public benchmark datasets: MMLU-Pro (Advanced Multi-Task Language Understanding), SuperGPQA (Super Graduate-Level Question Answering Dataset), and AGI-Eval (General Artificial Intelligence Evaluation Dataset). After processing, a training set of 9136 samples (for model parameter optimization) and a validation set of 956 samples (for domain adaptability verification and parameter tuning) are obtained, which can support multi-step reasoning ability training and multi-disciplinary knowledge coverage. The generalization test set outside the domain is an independent dataset that was not used in the training, containing 1000 samples each from SciQ (Science Question Answering Dataset), ScienceQA (Science Multiple Choice Question Dataset), and SciEval (Science Evaluation Dataset), used to verify the stability and adaptability of the model's reasoning in unfamiliar scenarios and cross-disciplinary domains.

[0031] Then, the data sampling and processing process is carried out.

[0032] To meet the requirements of the two-stage optimization, the data sampling strategy combines stratified sampling with distillation enhancement. This combined strategy ensures both the diversity of training data in terms of feature dimensions and improves the semantic quality of the data, providing high-quality training materials for the two-stage optimization.

[0033] 1. SFT stage: Inference distillation data construction.

[0034] The sampling range was stratified by subject from subsets of the datasets in the three domains: MMLU-Pro, SuperGPQA, and AGI-Eval. Approximately 167 samples were drawn from each dataset, for a total of 500 samples.

[0035] In this stage, the DeepSeek-R1 model is used to generate inference trajectories. The temperature is set to 1.0 to ensure the diversity of trajectories. The generated samples include three parts: scientific questions, multi-step inference trajectories, and final answers, which are used as distillation samples.

[0036] All reasoning trajectories must adhere to a unified structured format, using... <reasoning>The reasoning process is wrapped in labels, and the final answer is labeled to ensure the accuracy of subsequent reward calculations and length statistics. The generated 500 distillation samples constitute the distillation training set, which is used for training in the SFT stage, enabling the model to initially master multi-step reasoning and structured output capabilities. The output model serves as the initialization and reference model for subsequent reinforcement learning.

[0037] 2. GRPO training phase.

[0038] This phase employs a stratified sampling method based on question type to ensure a uniform distribution of samples across disciplines and difficulty levels within the training set. The data undergoes the following preprocessing steps: (1) Text standardization: Unified encoding is UTF-8, invisible characters are removed, excessively long clauses are reasonably truncated, and key steps of reasoning are retained.

[0039] (2) Quality filtering: Through dual verification of rules and models, samples with contradictory reasoning logic and incorrect answers are removed, and the final sample pass rate is ≥98%.

[0040] (3) Format uniformity: GPT-4 tokenizer (cl100k_base) is used to calculate the inference length to ensure that the length statistics of different models are comparable.

[0041] The final output of this stage is a structured scientific reasoning dataset that has undergone text standardization, quality filtering, and format unification preprocessing. The resulting dataset is comprehensive and well-structured, consisting of several subsets: a training set of 9136 samples (for iterative optimization of model parameters), a validation set of 956 samples (for monitoring overfitting and performance fluctuations during training) for parameter optimization, and a test set of 3367 samples for evaluating the model's final performance within the training distribution. Furthermore, to assess generalization ability, three out-of-domain datasets—SciQ, ScienceQA, and SciEval—each containing 1000 samples were constructed. All data are stored as triples of scientific question q-inference trajectory r-answer a, and the inference length is uniformly calculated using a GPT-4 tokenizer (cl100k_base) to ensure statistical comparability across model lengths. This dataset partitioning ensures the effectiveness of model training and verifies the model's robustness from multiple dimensions.

[0042] The SynergiSci framework's two-stage optimization both use this structured scientific reasoning dataset as their core input. Stage 1 (information gain-guided quality learning) uses the training set for inference distillation SFT and GRPO policy updates. First, it completes the model's cold start using distillation samples (500 samples, sampled from a subset of the training set). Then, it optimizes the inference quality using the full training set, while simultaneously monitoring training stability in real time using the validation set to avoid inference logic deviations. Stage 2 (dynamic length-controlled efficiency learning) also optimizes the inference length policy based on the training set, penalizing only redundant inferences exceeding a dynamic threshold. Finally, it evaluates the quality-efficiency balance of the optimized framework using the test set and verifies the framework's robustness in unfamiliar scientific reasoning scenarios using an out-of-domain dataset, achieving a closed-loop connection between data-supported framework optimization and data-validated framework performance.

[0043] The SFT phase is the foundation for the cold start of the SynergiSci framework, while the GRPO training phase is the core optimization vehicle of the framework. The SFT phase constructs high-quality distilled data (500 samples, with inference trajectories generated by DeepSeek-R1) to fine-tune all parameters of the backbone model (Qwen2.5-7B / LLaMA3.1-8B). The output reference model π_SFT provides the SynergiSci framework with an initial high-quality inference template, avoiding optimization failure in the first phase of the framework due to a lack of basic inference capabilities. The stratified sampling and data preprocessing process in the GRPO training phase provides standardized, high-quality training data for the two-stage optimization of the framework, ensuring that the framework optimization has reliable data support.

[0044] The SFT phase corresponds to the initialization stage of the SynergiSci framework: the reference model π_SFT output by SFT is directly used as the initialization model for stage 1 of the SynergiSci framework, and also as the reference model π_ref for stage 1 GRPO optimization. The policy update direction is constrained by KL divergence regularization (β=0.005) to prevent the inference quality from deviating from the baseline. The GRPO training phase corresponds to the policy optimization stage of the SynergiSci framework: stage 1 (quality learning) optimizes the information gain-guided reward through the GRPO algorithm, allowing the model to learn the inference process that improves the confidence of the answer. Stage 2 (efficiency learning) optimizes the dynamically length-controlled reward through the GRPO algorithm, pruning redundant inference while maintaining quality. That is, the algorithm logic of the GRPO training phase is the core means for the framework to realize policy updates in two stages.

[0045] Phase 1: Information gain-guided quality learning.

[0046] The core objective of this stage is to build high-quality scientific reasoning capabilities for the model—enabling it to master multi-step deduction paradigms that conform to scientific logic. This allows the model to decompose problems, invoke knowledge, and gradually deduce rigorous and reliable reasoning processes, laying a solid foundation for subsequent efficiency optimization. The core idea is to quantify the contribution of the reasoning process to the correctness of the final answer (i.e., information gain), and design a differentiated reward mechanism to prioritize learning reasoning trajectories that effectively improve the reliability of the answer.

[0047] In one embodiment, the information gain-guided quality learning phase specifically includes: (1) A set number of scientific questions are sampled from the scientific reasoning benchmark dataset using stratified sampling, and a pre-trained large language model is used to generate distillation samples containing multi-step reasoning trajectories and final answers for each scientific question, forming a distillation dataset.

[0048] In this embodiment, typical questions are selected from scientific reasoning benchmark datasets (such as MMLU-Pro, SuperGPQA, etc.), and large models (such as DeepSeek-R1) are used to generate detailed multi-step reasoning trajectories and labeled data of the final answer for each question (i.e. reasoning distillation data).

[0049] For example, for problems involving applications of the ideal gas law, a complete reasoning chain is formed: generating explicit known conditions → selecting the formula PV=nRT → substituting parameters for calculation → verifying unit consistency.

[0050] (2) The backbone model is supervisedly fine-tuned using the distillation dataset to obtain a reference model; the backbone model is either the Qwen2.5-7B model or the LLaMA3.1-8B model.

[0051] In this embodiment, these distillation data are used to perform supervised fine-tuning of the backbone model (such as Qwen2.5-7B, LLaMA3.1-8B) to train a reference model capable of generating basic canonical reasoning (denoted as ). This provides a benchmark template for high-quality reasoning in subsequent reinforcement learning.

[0052] The specific adjustment process is as follows: Inference distillation dataset construction: 500 typical scientific reasoning questions (approximately 167 samples per dataset) were stratified by subject from subsets of MMLU-Pro, SuperGPQA, and AGI-Eval. Using the DeepSeek-R1 model (with temperature set to 1.0 to ensure diversity of reasoning trajectories), labeled data of multi-step reasoning trajectories r + final answer a were generated for each question, forming the distillation dataset. Furthermore, all reasoning trajectories must adhere to a unified structured format (using...). <reasoning>The reasoning process is wrapped in tags (with the answer labeled with [ANSWER]) to ensure consistency in formatting during subsequent fine-tuning.

[0053] Reference model initialization fine-tuning: Using Qwen2.5-7B-Instruct or LLaMA3.1-8B-Instruct as the backbone model, the SFT stage is performed on D_distill using a full-parameter fine-tuning approach. The optimization objective is to maximize the generation probability of the inference sequence, and the core loss function is: .

[0054] Where (q,r,a) represents In the triplet sample, y=[r;a] is the concatenated sequence of the inference chain and the answer, and T is the total number of tokens for y. This represents the probability that the model generates the t-th token given a question q and the first t-1 tokens. During fine-tuning, DeepSpeed ​​ZeRO Stage 2 is used to optimize distributed training. The maximum cue length and generation length are both set to 4096 tokens, the learning rate is 5e-5, the linear warmup strategy is used (the warmup step accounts for 10% of the total steps), the Adam optimizer has β1=0.9 and β2=0.99, the weight decay coefficient is 0.1, and training is continued until the loss converges (usually iterated until the loss fluctuation on the validation set is <0.01).

[0055] Reference model determination: The output model after fine-tuning That is, the reference model The model possesses basic, standardized scientific reasoning capabilities; however, for subsequent GRPO optimization, its stability still needs to be fixed. The parameters, in the GRPO update of framework phase 1, are constrained by KL divergence regularization (β=0.005) between the current inference trajectory generation model and... To avoid deviations and excessive shifts in reasoning logic.

[0056] (3) For the scientific question, the first confidence level of the reference model when directly outputting the answer and the second confidence level when outputting the answer after generating the reasoning trajectory are obtained respectively, and the difference between the second confidence level and the first confidence level is calculated as the information gain.

[0057] In this embodiment, for each scientific question q, the model will produce two types of outputs: one is a direct answer, which gives the answer directly without reasoning, and the confidence of the model in the answer can be measured by the logits of the output layer; the other is an answer after reasoning, which first generates the reasoning process and then gives the answer, and the confidence of the answer can also be obtained.

[0058] Information gain is defined as the difference between these two confidence levels, i.e.: Information gain = Logits after inference - Logits of direct response.

[0059] Where Logits is the logarithmic odds.

[0060] Intuitively, if the information gain is positive, it means that the reasoning process provides effective support for the answer and improves the model's confidence in the answer; if it is negative, it means that the reasoning process may contain redundancy or errors and provides limited help to the answer.

[0061] (4) Based on the information gain, output format compliance and answer correctness, construct a comprehensive reward function.

[0062] To guide the model in learning high-quality reasoning, a formatted reward system is designed. Accuracy Bonus Information gain reward The comprehensive reward, wherein the comprehensive reward function is: ; in, As a comprehensive reward; To ensure a consistent format, the output of the reasoning process conforms to a uniform structure (e.g., wrapping reasoning steps with <reasoning> tags, ending with < / reasoning>, and marking the answer with [ANSWER]). If the format is compliant, then... If the format is not compliant, then ; As an accuracy bonus, if the final answer matches the standard answer, then... (High weighting ensures the model prioritizes the reliability of the conclusions); if the final answer differs from the standard answer, then... ; The reward is designed based on the magnitude of the information gain. The information gain is mapped to the [0,1] interval using the Sigmoid function, and then multiplied by a coefficient. ,Right now: , For coefficients, , For temperature parameters, This controls the sensitivity of the sigmoid function to information gain. The greater the information gain, the more sensitive the sigmoid function becomes to information gain. The higher the value, the more likely the model will learn reasoning trajectories that are most helpful in answering the question.

[0063] The quantification result of information gain is directly used as the core input of information gain reward, realizing the transformation of the degree of contribution of the reasoning process to the answer into a reward signal. In conjunction with format and accuracy rewards, it guides the model to learn behaviors that are compliant in format, correct in answer and valuable in reasoning.

[0064] (5) Based on the comprehensive reward function and the reference model, the current inference trajectory generation model is updated using a group relative strategy optimization algorithm to obtain the updated inference trajectory generation model, specifically including: For each scientific question, sample G different inference trajectories from the current inference trajectory generation model; Calculate the overall reward for each inference trajectory, the average reward and standard deviation of the rewards for G inference trajectories, and then calculate the standardized within-group advantage for the i-th inference trajectory; Based on the standardized intra-group advantage of the i-th inference trajectory, the deviation between the current inference trajectory generation model and the reference model is constrained by KL divergence regularization. The parameters of the current inference trajectory generation model are updated by the objective function of the group relative strategy optimization algorithm, resulting in the updated inference trajectory generation model.

[0065] In this embodiment, the Group Relative Policy Optimization (GRPO) algorithm is used to update the model based on the comprehensive reward function. The specific process is as follows: 1. Trajectory Sampling: For each scientific question Generate a model from the current inference trajectory Medium sampling Different reasoning trajectories , For the first The reasoning process, This is the corresponding answer.

[0066] 2. Reward Calculation: Calculate the comprehensive reward for each trajectory. (Substitute) The formula yields the first... (Total reward for each trajectory).

[0067] 3. Within-group advantage calculation: First calculate the advantage of the current group ( Average reward of (trajectory) and the standard deviation of rewards Then calculate the first The advantages of standardized trajectories within a group: .

[0068] Where, add 10 to the denominator -8 Avoid calculation errors when the standard deviation is 0.

[0069] 4. Policy Gradient Update: Update model parameters using the GRPO objective function. The objective function is: ; in, Let be the objective function, representing the optimization objective under parameter θ; It is the expectation of scientific reasoning benchmark D; q represents the logarithmic probability of the inference trajectory generated by the current inference trajectory generation model, where q is the science question. For scientific question q A line of reasoning, For the first The final answer corresponding to each reasoning trajectory; Here is the KL divergence regularization coefficient; ; It is a reference model; This indicates that the KL divergence regularization constraint ensures that the current inference trajectory generation model does not deviate from the reference model.

[0070] By leveraging the strengths of each group to highlight the relative value of high-quality trajectories, and using KL regularization to ensure inference quality, we guide model parameter updates, making the model more inclined to generate inference trajectories with standardized formats, correct answers, and high information gain, thereby gradually strengthening high-quality inference capabilities.

[0071] Phase 2: Efficiency learning of dynamic length control.

[0072] This phase builds upon the high-quality reasoning model trained in Phase 1. Its core objective is to optimize reasoning efficiency without sacrificing reasoning rigor—by reducing redundant steps in the reasoning process, such as repetitive explanations and unnecessary intermediate derivations, thereby decreasing the number of tokens and computation time, while ensuring the correctness of the conclusion. Its key mechanism is the dynamic length threshold: adaptively determining a reasoning length standard that is both correct and sufficiently concise based on the complexity of the current problem. Phase 2 is deeply and closely logically connected to Phase 1, both serving the core objective of quality-efficiency synergy. Phase 1 provides a high-quality reasoning capability foundation for Phase 2. The model trained in Phase 1 using SFT, information gain quantization, and the GRPO algorithm has been internalized as a basic model with rigorous reasoning logic, providing a prerequisite for Phase 2 to optimize efficiency without sacrificing reasoning rigor and preventing a drop in accuracy due to a lack of core reasoning capabilities. Phase 1 also provides a reliable reference for the dynamic length threshold in Phase 2. The criteria for determining the correct answer in Phase 2 are consistent with the accuracy reward criteria in Phase 1, and the high-quality correct reasoning trajectories generated in Phase 1 provide a basis for calculating the dynamic threshold. Provides logically complete and non-redundant reference samples to ensure that the threshold setting meets the needs of scientific reasoning and can accurately locate the starting point of redundancy. At the same time, Phase 2 adopts the GRPO algorithm of Phase 1. On the basis of inheriting the logic of advantage comparison within the group, it adjusts the reward focus in combination with the efficiency optimization goal to ensure smooth iteration of model parameters and avoid parameter oscillation caused by sudden changes in optimization methods. In addition, the two complement each other by decomposing the goal of building quality first and then optimizing efficiency. Phase 1 focuses on enabling the model to reason correctly and solves the problem of sloppy reasoning logic. Phase 2 focuses on enabling the model to reason quickly and cost-effectively and solves the problem of high computational overhead. Without either phase, the synergistic effect of improving accuracy and reducing inference length cannot be achieved.

[0073] In one embodiment, the efficiency learning phase of dynamic length control specifically includes: (1) For each scientific question in the current batch, select the correct reasoning trajectory from several reasoning trajectories generated by the model, and use the shortest reasoning length as the dynamic length threshold of the corresponding scientific question.

[0074] Based on the synergistic effect of improved accuracy and reduced inference length, a dynamic length threshold is calculated using a multi-dimensional weighted fusion method: For each scientific question q, after the model generates G inference trajectories, it first selects the trajectories with the correct answer; then, the shortest inference length among these correct trajectories is used as the dynamic length threshold. For example, if the lengths of the three correct inference trajectories for a certain question are 200 Tokens, 180 Tokens, and 220 Tokens respectively, then the dynamic length threshold is... Take 180 Tokens.

[0075] If there is no correct trajectory in the current batch (which is extremely rare and usually occurs in very complex problems), then the average length of all trajectories in the current batch multiplied by 0.8 is taken as the neutral baseline threshold. This is to avoid inaccurate thresholds due to the lack of a correct reference.

[0076] (2) Construct an efficiency reward function that includes an accuracy reward and a length penalty term; wherein the length penalty term is determined based on the current inference trajectory length and the dynamic length threshold.

[0077] In one embodiment, based on the dynamic length threshold obtained in step (1) or We designed a reward mechanism that prioritizes accuracy and penalizes length using a multi-dimensional weighted fusion method. The efficiency reward function is as follows: ; in, Reward for efficiency; This is the adjustment coefficient; The length of the currently generated inference trail; This is the dynamic length threshold.

[0078] In this setup, the reward value was set at 3.0. This value was determined based on an in-depth analysis and reference to the accuracy reward weighting in Phase 1. The purpose of this is to ensure that the correctness of the answer remains the most critical priority during the efficiency optimization phase. This approach effectively prevents the model from unintentionally sacrificing the rigor and accuracy of the reasoning process in the pursuit of shorter answer lengths, thus ensuring that the output answer is not only concise and efficient in form but also maintains a high degree of accuracy and logical rigor in content.

[0079] (3) Based on the efficiency reward function, the updated reasoning trajectory generation model is updated using the group relative strategy optimization algorithm to obtain the scientific reasoning model.

[0080] In this embodiment, the updated model is updated using the GRPO algorithm based on the efficiency reward function obtained in (2) above: The updated inference trajectory generation model is also updated using the GRPO algorithm. First, it ensures that the updated model prioritizes correct answer generation. Building on this, penalties are applied to address redundancy exceeding a threshold, guiding the model to proactively control the length of the inference process within a reasonable range while ensuring correct reasoning. However, the training process focuses more on balancing inference length and correctness. The model gradually learns: for simple problems, such as basic concept differentiation, to quickly arrive at the correct conclusion using shorter inference chains; for complex problems, such as physical derivations involving multiple formulas, to retain necessary key steps to ensure correctness while pruning redundant explanatory text. Through this adaptive adjustment, inference length is dynamically matched to problems of different complexities, improving overall efficiency.

[0081] The two phases of the SynergiSci framework are not independent of each other, but rather achieve deep collaboration through reference model propagation and decoupling of optimization objectives: Reference model transfer: The high-quality reference model trained in Phase 1 provides a baseline for the inference quality of Phase 2. The model initialization in Phase 2 is based on this reference model to avoid destroying the core inference logic when optimizing efficiency and to ensure that the rigor of the inference is always online.

[0082] The optimization objectives are decoupled: Phase 1 focuses on laying a quality foundation, enabling the model to reason correctly; Phase 2 focuses on efficiency optimization, enabling the model to reason quickly and efficiently. This avoids the conflict between the two objectives that can lead to compromises when optimizing both quality and efficiency simultaneously in a single phase. After completing information gain-guided quality learning in Phase 1, the core parameters of the trained high-quality reference model are selected. Key parameters that determine the scientific reasoning logic, such as the Transformer encoder attention layer weights, Feed-Forward network weights, and word embedding layer weights, are retained to form a reference model parameter set. At the same time, the output inference quality benchmark indicators (such as in-domain inference accuracy, average inference information gain) and average length of the correct inference trajectory are also provided. When performing efficiency learning with dynamic length control in Phase 2, the core inference parameters are first initialized by loading them directly into the core model. Then, the core inference parameters are frozen (only the output layer adaptation parameters related to inference length control are unfrozen) to ensure that the core inference logic built in Phase 1 is not destroyed when optimizing efficiency. This is the parameter-level transfer coupling. In optimizing the decoupled computation of the objective, Stage 1 focuses on quality foundation through a comprehensive reward function of format reward + accuracy reward + information gain reward. Stage 2, based on the accuracy judgment criteria of Stage 1, designs a comprehensive reward function of accuracy reward - length penalty, with trajectory length as the dynamic threshold to focus on efficiency optimization. The dynamic threshold calculation in Stage 2 refers to the reasonable range determined by the output of Stage 1. The GRPO algorithm also continues the intra-group advantage comparison logic of Stage 1 and adds a length deviation term to optimize the objective. Finally, through such parameter transfer, computational logic reuse, and objective decoupling, the two-stage progressive training outputs a scientific inference model that combines high inference quality (57.2% in-domain accuracy and 96.2% out-of-domain accuracy) and high efficiency (28.6% reduction in inference length within the domain and a reduction in single inference time from 0.8s to 0.5s on the NVIDIA A800 GPU), achieving a synergistic improvement in quality and efficiency.

[0083] Ultimately, through a two-stage progressive training process, the model can generate logically rigorous and reliable scientific reasoning, meeting the accuracy requirements of scientific research, while also dynamically adjusting the reasoning length according to the complexity of the problem, reducing computational costs, and meeting the efficiency requirements of engineering, thus truly achieving a synergistic improvement in quality and efficiency.

[0084] This application proposes a two-stage scientific reasoning quality-efficiency co-optimization framework (SynergiSci), such as... Figure 2 As shown, the dual-stage collaborative architecture of quality learning guided by information gain and efficiency learning with dynamic length control solves the three core challenges in scientific reasoning: the difficulty in balancing quality and efficiency, the high redundancy of the reasoning process, and the insufficient adaptability of reasoning for complex problems.

[0085] The SynergiSci two-stage optimization method proposed in this application demonstrates significant advantages in terms of quality, efficiency, and engineering feasibility compared to existing scientific reasoning models, and each advantage directly corresponds to the core technology design.

[0086] Regarding the synergistic improvement of quality and efficiency, this method overcomes the limitation of existing models where quality and efficiency cannot be simultaneously achieved. On the one hand, the average accuracy within the domain increases from 54.4% to 57.2%, and the average accuracy outside the domain jumps from 76.3% to 96.2%. Particularly in the medical inference dataset SciEval, the accuracy improves from 83.4% to 86.1%, approaching the level of the commercial model o4-mini. This precision advantage stems from the information gain quantification mechanism—by comparing the difference in logits between models that provide direct answers and those that provide answers with reasoning processes, the contribution of reasoning steps to reducing answer uncertainty is precisely quantified. This guides the model to prioritize learning rigorous and valuable reasoning logic (such as physical formula derivation and medical symptom-cause association), reducing the need for complex reasoning. The model avoids logical fallacies such as formula misuse and causal reversal. On the other hand, the average inference length within the domain decreased from 714 tokens to 445 tokens (a reduction of 28.6%), and outside the domain from 407 tokens to 234 tokens (a reduction of 42.5%). Furthermore, for complex tasks (such as SuperGPQA graduate-level inference), the length was reduced by only 15.6% to preserve inference depth. This efficiency optimization is attributed to the adaptive length threshold design—using the shortest length of correct inference in the current batch as a dynamic standard, only penalizing excessively long inferences exceeding the threshold, avoiding inference breakage caused by a one-size-fits-all constraint. Simultaneously, the two-stage hierarchical training strategy first establishes a high-quality inference foundation through SFT+GRPO, and then optimizes efficiency, further ensuring that quality and efficiency do not conflict. In addition, the model's accuracy on outside datasets (such as ScienceQA academic literature inference) improved from 58.0% to 74.8%, far exceeding the open-source baseline. This enhanced generalization ability also stems from the two-stage hierarchical training strategy: the first stage constructs a general inference logic through multidisciplinary data, and the second stage uses dynamic thresholds to adapt to tasks of different complexities, enabling it to handle unseen cross-disciplinary or open-domain scenarios.

[0087] In terms of practical application, this method possesses unparalleled utility compared to existing technologies: First, it boasts strong compatibility, adapting to mainstream open-source models such as Qwen2.5-7B and LLaMA3.1-8B. It requires no architecture reconstruction and can be optimized solely through LoRA fine-tuning, reducing training costs by 60%. This advantage stems from the method's design's consideration of model adaptability, avoiding binding to specific model architectures, and the LoRA fine-tuning method reduces resource consumption during full-parameter training. Second, it offers high interpretability, employing [a specific method / technology] in the inference process. <reasoning>With the structured label [ANSWER], and by tracing the reasons for retention / pruning at each step of inference through the information gain quantization mechanism, the black-box inference problem of existing models is solved. This is directly related to the quantification of the value of the inference process by the information gain quantization mechanism. Thirdly, it has low resource consumption. On the NVIDIA A800 GPU, the time for a single inference is reduced from 0.8s to 0.5s, supporting high-concurrency scenarios. This is due to the adaptive length threshold design that reduces redundant inference steps and reduces computational overhead. At the same time, the two-stage hierarchical training avoids resource waste caused by ineffective optimization.

[0088] Furthermore, this method uses multi-dimensional evaluation fusion (combining accuracy, inference length, and AES score) to form a comprehensive optimization standard, avoiding the one-sidedness of existing technologies that rely on a single indicator. This ensures that the model can guarantee both inference accuracy and efficiency requirements in practical applications, further highlighting its superiority over single-objective optimization or fixed-constraint schemes.

[0089] There are three potential alternatives in the existing technology, but all of them cannot achieve the same effect due to design flaws. The specific analysis is as follows: The first type is the fixed-length constraint scheme. Its core idea is to set a uniform length threshold (e.g., 300 tokens) for all scientific reasoning tasks, and forcibly prune reasoning content exceeding the threshold. The flaws of this scheme are obvious: for simple scientific QA tasks (such as basic scientific concept judgment), reasoning that could be completed with 100 tokens is forcibly extended to 300 tokens, causing serious length redundancy; while for complex reasoning tasks (such as multi-step physical formula derivation and medical pathology analysis), if the reasoning reaches the threshold and is pruned at 250 tokens, it will directly lead to logical breakage and prevent the correct answer from being obtained. In contrast, the adaptive length threshold design of this application can dynamically adjust the standard according to the complexity of the problem, using the shortest length of the correct reasoning in the current batch as the threshold, achieving fast solutions for simple problems and deep reasoning for complex problems, avoiding redundant calculations without sacrificing reasoning depth. The second type is the single-stage reward fusion scheme. This scheme directly merges quality rewards and efficiency rewards into a single objective function, achieving a balance between quality and efficiency through synchronous optimization. However, this design has two core problems: First, there is a conflict of objectives. Quality rewards essentially encourage the model to generate more complete and longer inference chains to ensure rigor, while efficiency rewards require the model to shorten the inference length. The two are mutually restrictive, leading to instability in the policy update process. Second, there is reward dilution. The information gain signal of the inference process (i.e., the contribution of the inference steps to reducing the uncertainty of the answer) is masked by the length penalty, and the model cannot accurately learn high-quality inference patterns. This application avoids this defect through a two-stage hierarchical training strategy. First, a high-quality inference foundation is established through supervised fine-tuning (SFT) and GRPO reinforcement learning, and then efficiency optimization is performed, completely avoiding the conflict of optimization objectives. At the same time, the signal of the information gain quantification mechanism is made clearer, ensuring that the model prioritizes learning valuable inference steps. The third type is the model distillation and compression scheme, which attempts to reduce resource consumption while maintaining accuracy by distilling the inference capabilities of high-parameter, high-quality models (such as DeepSeek-R1-70B) into low-parameter models (such as Qwen2.5-7B). However, this approach has significant limitations: firstly, there is a noticeable loss of accuracy during the distillation process; for example, on the MMLU-Pro dataset, the accuracy dropped from 58.3% for the large model to 43.13% for the distilled small model, a decrease of over 5%; secondly, efficiency optimization only reduces the number of model parameters and does not address the redundancy of the inference steps themselves, resulting in a large amount of ineffective computation. This application, through the synergistic effect of information gain quantization and dynamic length control, achieves a dual gain in accuracy and efficiency—improving accuracy by 3.5% and reducing inference length by 28.6% across six major scientific inference benchmarks, completely overcoming the trade-off between these two approaches.In summary, existing alternatives fail to resolve the core contradiction of synergistic optimization between quality and efficiency in scientific reasoning; they either sacrifice accuracy for efficiency or prioritize accuracy over efficiency. The two-stage framework proposed in this application, through its hierarchical design prioritizing quality over efficiency, its information gain quantification mechanism, and its adaptive control of dynamic thresholds, forms an irreplaceable technical solution. It not only achieves a synergistic effect of improved accuracy and reduced length in six major benchmark tests but also further reduces engineering implementation costs by being compatible with mainstream open-source models and lowering training and inference resource consumption, giving it a unique technological advantage.

[0090] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described scientific reasoning method based on a scientific reasoning model.

[0091] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described scientific reasoning method based on a scientific reasoning model.

[0092] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described scientific reasoning method based on a scientific reasoning model.

[0093] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a scientific reasoning method based on a scientific reasoning model.

[0094] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0096] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0097] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0098] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0099] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.< / reasoning> < / reasoning> < / reasoning>

Claims

1. A scientific reasoning method based on a scientific reasoning model, characterized in that, include: Obtain the scientific question to be reasoned; Based on the scientific question to be reasoned about, the reasoning result is determined using a scientific reasoning model; The scientific reasoning model is obtained by training and optimizing the reasoning trajectory generation model using a scientific reasoning benchmark dataset through a quality learning phase guided by information gain and an efficiency learning phase with dynamic length control. The scientific reasoning benchmark dataset includes the MMLU-Pro dataset, the SuperGPQA dataset, and the AGI-Eval dataset.

2. The scientific reasoning method based on a scientific reasoning model according to claim 1, characterized in that, The information gain-guided quality learning phase specifically includes: A set number of scientific questions are sampled from the scientific reasoning benchmark dataset using stratified sampling, and a pre-trained large language model is used to generate distillation samples containing multi-step reasoning trajectories and final answers for each scientific question, forming a distillation dataset; The backbone model was subjected to supervised fine-tuning using the distillation dataset to obtain a reference model; the backbone model was either the Qwen2.5-7B model or the LLaMA3.1-8B model. For the scientific question, the first confidence level of the reference model when directly outputting the answer and the second confidence level when outputting the answer after generating the inference trajectory are obtained respectively, and the difference between the second confidence level and the first confidence level is calculated as the information gain; Based on the aforementioned information gain, output format compliance, and answer correctness, a comprehensive reward function is constructed; Based on the comprehensive reward function and the reference model, the current inference trajectory generation model is updated using a group relative strategy optimization algorithm to obtain the updated inference trajectory generation model.

3. The scientific reasoning method based on a scientific reasoning model according to claim 2, characterized in that, The comprehensive reward function is: ； in, As a comprehensive reward; A reward will be given based on the format; if the format is compliant, then... If the format is not compliant, then ; A reward is given for accuracy; if the final answer matches the standard answer, then... If the final answer differs from the standard answer, then ; Rewards are given for information gain. , For coefficients, , For temperature parameters, .

4. The scientific reasoning method based on a scientific reasoning model according to claim 2, characterized in that, Based on the comprehensive reward function and the reference model, the current inference trajectory generation model is updated using a group relative policy optimization algorithm to obtain the updated inference trajectory generation model, specifically including: For each scientific question, sample G different inference trajectories from the current inference trajectory generation model; Calculate the total reward for each inference trajectory, the average reward and standard deviation of the rewards for G inference trajectories, and then calculate the standardized within-group advantage for the i-th inference trajectory; Based on the standardized intra-group advantage of the i-th inference trajectory, the deviation between the current inference trajectory generation model and the reference model is constrained by KL divergence regularization. The parameters of the current inference trajectory generation model are updated by the objective function of the group relative strategy optimization algorithm, resulting in the updated inference trajectory generation model.

5. The scientific reasoning method based on a scientific reasoning model according to claim 4, characterized in that, The objective function is: ； in, The objective function is... It is the expectation of scientific reasoning benchmark D; q represents the logarithmic probability of the inference trajectory generated by the current inference trajectory generation model, where q is the science question. For scientific question q A line of reasoning, For the first The final answer corresponding to each reasoning trajectory; Here is the KL divergence regularization coefficient; ; It is a reference model; This indicates that the KL divergence regularization constraint ensures that the current inference trajectory generation model does not deviate from the reference model.

6. The scientific reasoning method based on a scientific reasoning model according to claim 1, characterized in that, The efficiency learning phase of dynamic length control specifically includes: For each scientific question in the current batch, the correct reasoning trajectory is selected from several reasoning trajectories generated by the updated reasoning trajectory generation model, and the shortest reasoning length among them is used as the dynamic length threshold for the corresponding scientific question. Construct an efficiency reward function that includes an accuracy reward and a length penalty term; wherein the length penalty term is determined based on the current inference trajectory length and the dynamic length threshold; Based on the efficiency reward function, the updated reasoning trajectory generation model is updated using a group relative strategy optimization algorithm to obtain a scientific reasoning model.

7. The scientific reasoning method based on a scientific reasoning model according to claim 6, characterized in that, The efficiency reward function is: ； in, Reward for efficiency; This is the adjustment coefficient; The length of the currently generated inference trail; This is the dynamic length threshold.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the scientific reasoning method based on a scientific reasoning model according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the scientific reasoning method based on the scientific reasoning model as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the scientific reasoning method based on the scientific reasoning model as described in any one of claims 1-7.

Images