Scientific research reasoning task processing method, equipment and medium

Abstract

The embodiment of the invention discloses a scientific research inference task processing method and device and a medium, and relates to the technical field of large models.The method comprises the steps that a scientific research inference task request is received, the scientific research inference task request comprises an input text sequence, and the input text sequence is from a scientific literature database or scientific research query submitted by a user; processing the input text sequence through a pre-loaded pre-training reasoning large model, and performing long-sequence attention optimization on the input text sequence of which the sequence length exceeds a preset threshold value to generate final attention output; based on the final attention output, a scientific research reasoning result is generated through a reasoning layer of the pre-training reasoning large model, and the scientific research reasoning result comprises text implication judgment, common sense reasoning answers or logic structure analysis output. And the scene requirements of common large-scale literature analysis, complex problem solving and other practical application scenes in the scientific research environment can be met.

Description

Technical Field

[0001] This specification relates to the field of large model technology, and in particular to a method, device and medium for processing scientific research reasoning tasks. Background Technology

[0002] Against the backdrop of the rapid development of artificial intelligence technology, scientific reasoning task processing systems face multiple technical challenges. These tasks involve lengthy scientific documents with complex terminology and logic, or scientific problems requiring multi-step reasoning. Current technologies, such as large language model-based reasoning systems, typically employ end-to-end full-parameter training, revealing significant technical limitations when handling complex scientific tasks.

[0003] In terms of model architecture, existing systems generally adopt a uniform Transformer structure to process various types of scientific texts, failing to fully consider the differences between basic language understanding and advanced reasoning capabilities. This architecture results in insufficient semantic understanding depth when processing scientific literature containing technical terms and complex logic, making it difficult to accurately capture the implicit logical relationships in scientific texts. Regarding sequence processing capabilities, traditional attention mechanisms employ a fully connected computational model. When processing long scientific documents or scientific questions involving multi-step reasoning, the secondary increase in computational complexity leads to a sharp rise in memory consumption, causing response delays or even processing interruptions when processing scientific texts exceeding a certain length. During model training, existing training methods use a fixed learning rate scheduling strategy. This strategy cannot adapt to the dynamic changes during the training process of scientific reasoning tasks, easily causing parameter oscillations in the early stages of training and making fine-tuning difficult in the later stages. Single-task training modes lead to overfitting of specific types of scientific questions, resulting in a significant decline in performance on interdisciplinary reasoning tasks. Fixed batch size training strategies fail to consider the differences in scientific samples of varying complexity; simple samples are inefficient for training, while complex samples are prone to gradient anomalies.

[0004] Therefore, in the process of scientific research reasoning tasks, due to the limitation of the computational complexity of the attention mechanism, there is a problem of low efficiency in processing long sequences of scientific research texts. Furthermore, the training strategy lacks dynamic adaptability, the model converges slowly and has weak generalization ability, resulting in unstable accuracy and limited applicability in the scientific research reasoning process in practical applications. Summary of the Invention

[0005] This specification provides one or more embodiments of a scientific research reasoning task processing method, device, and medium to solve the following technical problems: In the process of scientific research reasoning task processing, due to the limitation of the computational complexity of the attention mechanism, there is a problem of low efficiency in processing long sequences of scientific research texts, and the training strategy lacks dynamic adaptability, the model convergence speed is slow and the generalization ability is weak, resulting in the problem of unstable accuracy and limited applicability of the scientific research reasoning process in practical applications.

[0006] One or more embodiments of this specification employ the following technical solutions: This specification provides one or more embodiments of a scientific research reasoning task processing method, the method comprising: receiving a scientific research reasoning task request, wherein the scientific research reasoning task request includes an input text sequence, the input text sequence being derived from a scientific literature database or a scientific research query submitted by a user; processing the input text sequence through a pre-loaded pre-trained large-scale reasoning model, performing long-sequence attention optimization on input text sequences whose sequence length exceeds a predetermined threshold to generate a final attention output; and generating a scientific research reasoning result based on the final attention output through the reasoning layer of the pre-trained large-scale reasoning model, wherein the scientific research reasoning result includes textual implication judgments, common sense reasoning answers, or logical structure parsing outputs.

[0007] This specification provides one or more embodiments of a scientific research reasoning task processing device, including: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the above-described method.

[0008] This specification provides one or more embodiments of a non-volatile computer storage medium storing computer-executable instructions configured to perform the above-described method.

[0009] The at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: By constructing an end-to-end processing flow for scientific reasoning tasks, introducing a long sequence attention optimization mechanism, and adopting a hybrid computing mode that integrates local window attention and global sparse attention, a precise balance is achieved between computational complexity and semantic preservation. This allows the model to maintain sensitivity to local details and the ability to understand the global logical structure when processing long texts such as scientific literature and experimental reports, thus breaking through the structural limitations of traditional models in long sequence processing at the principle level. The reasoning layer adopts a reasoning-oriented attention mechanism, which explicitly models reasoning elements such as causal relationships, evidence support relationships, and conditional dependencies in the text by constructing a logical relationship graph and multi-round iterative information transmission. This enables the model to go beyond simple word co-occurrence statistics and deeply understand the implicit logical context and argumentation structure in scientific texts. Traditional methods often require the deployment of independent model systems for different reasoning tasks, resulting in redundant computing resources and increased maintenance costs, and failing to fully utilize the correlation between tasks. The technical solution in this specification achieves positive knowledge transfer between different inference tasks while maintaining model parameter efficiency through a shared underlying encoder and a dedicated multi-task inference layer. The pre-trained model loading mechanism and dynamic optimization strategy ensure the system's practicality and stability in real-world deployment environments. Parameter sharing in the pre-trained model significantly reduces computational overhead during inference, while the dynamic optimization mechanism for long sequence processing enables the system to adaptively adjust its computational strategy based on input features, maximizing resource utilization efficiency while ensuring result quality. This design allows the technical solution to meet the needs of common research environments such as large-scale literature analysis and complex problem solving. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart illustrating a scientific reasoning task processing method provided in the embodiments of this specification; Figure 2 This is a schematic diagram of the structure of a scientific research reasoning task processing device provided in the embodiments of this specification. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0012] This specification provides a method for processing scientific reasoning tasks. It should be noted that the executing entity in this specification can be a server or any device with data processing capabilities. Figure 1 This is a flowchart illustrating a scientific reasoning task processing method provided in an embodiment of this specification, such as... Figure 1 As shown, the main steps include the following: Step S101: Receive a scientific reasoning task request.

[0013] The research reasoning task request includes an input text sequence, which comes from a scientific literature database or a research query submitted by the user. In one embodiment of this specification, when a user submits a scientific research query request through a web interface or API interface, a request data packet containing an input text sequence is received via HTTP / HTTPS protocol. The text content and metadata information in the message payload are extracted. The sources of the input text sequence are divided into two categories. One category is structured text data obtained from scientific literature databases through a database connection module. This data is then connected to a knowledge base server via a JDBC / ODBC interface to execute predefined SQL query statements to extract target text paragraphs from professional databases such as scientific paper abstract databases and patent literature databases. The other category is natural language query statements directly entered by the user through an interactive interface. The text acquisition module receives the description of the scientific research question submitted by the user and performs encoding format conversion and character set unification processing on it.

[0014] The received input text sequence then enters the preprocessing process. First, the text cleaning submodule removes irrelevant control characters and formatting tags. Next, the language detection submodule identifies the language of the text. Then, the word segmentation submodule converts the continuous text into a token sequence that meets the model's input requirements. Finally, the sequence normalization submodule aligns the text length to ensure that the output conforms to the standard input format of the pre-trained inference model. The normalized text sequence after preprocessing will be temporarily stored in a memory buffer, waiting for subsequent calls from the model inference module.

[0015] Compared to traditional scientific research information processing systems, conventional systems typically use a single data source access method, which cannot simultaneously adapt to mixed scenarios of structured database queries and unstructured user input, thus limiting the system's applicability. However, the embodiments in this specification achieve standardized processing of multi-source heterogeneous scientific research data by constructing a unified task request receiving pipeline. This supports both accurate extraction of target text from professional literature databases and effective processing of user natural language queries, significantly improving the system's practicality and coverage.

[0016] Step S102: The input text sequence is processed by a pre-loaded pre-trained inference large model. For input text sequences whose sequence length exceeds a predetermined threshold, long sequence attention optimization is performed to generate the final attention output.

[0017] Before processing the input text sequence using a pre-loaded, pre-trained large inference model, the method further includes: training a base layer based on a pre-acquired public text dataset corpus to perform a language modeling task, wherein the base layer includes a word embedding layer and a basic Transformer encoder; acquiring a domain-specific research corpus dataset based on a pre-acquired scientific literature abstract database to train a feature layer using the domain-specific research corpus dataset to perform a semantic matching task; freezing specified parameters of the base layer during the training of the feature layer, wherein the percentage of frozen parameters is determined based on the training convergence of the base layer; and training an inference layer using a research inference task corpus from a pre-acquired scientific question dataset to perform an inference prediction task, wherein all parameters of the base layer and specified parameters of the feature layer are frozen during the training of the inference layer.

[0018] In one embodiment of this specification, the large-scale reasoning model is functionally layered, trained in stages, and the low-level parameters are gradually frozen to reduce redundant computation. The low-level (base layer) is responsible for general language capabilities and does not require frequent updates after training; the high-level (feature layer, reasoning layer) is responsible for task-specific capabilities and only updates some parameters to reduce computational consumption. The large-scale reasoning model includes a base layer, a feature layer, and a reasoning layer. The base layer includes a word embedding layer that maps tokens to vectors, and the first 3-6 layers of Transformer encoders that implement basic language modeling, outputting a general language representation. The feature layer includes the 7th-12th layers of Transformer encoders and a feature fusion layer that integrates contextual semantic features, outputting domain-adaptive semantic features. The reasoning layer includes a reasoning attention layer that focuses on logical connections and an output prediction layer that outputs reasoning results, such as answers and implication relationships, to achieve logical reasoning.

[0019] The training process is divided into stages. The base layer is trained using a general corpus (WikiText-103, approximately 10 billion tokens) for a language modeling task (predicting the next token). The optimizer is AdamW, with an initial learning rate of 2e-5, and 12 training epochs until the perplexity is below 15. The feature layer is trained using a domain-specific inference corpus (such as the scientific literature corpus PubMed Central, approximately 5 billion tokens) for a semantic matching task, determining whether two sentences are semantically consistent. 80% of the base layer's underlying parameters are frozen, with only the word embedding layer being fine-tuned. The learning rate is 1e-5, and 6 training epochs are performed until the semantic matching accuracy is above 88%. The inference layer is trained using a specialized inference corpus (GSM8K mathematical inference dataset, 100,000 math problems and solution steps) for an inference prediction task (outputting math problem answers). All parameters of the base layer and 60% of the feature layer's parameters are frozen. The learning rate is 5e-6, and 10 training epochs are performed until the math problem answering accuracy is above 75%.

[0020] The method further includes: during the training of the feature layer and the training of the inference layer, obtaining the change range of the training loss value corresponding to multiple training sessions to determine the loss fluctuation value; when the loss fluctuation value is determined to be less than a preset convergence threshold, it is determined to be converged to determine the training convergence degree; and based on the training convergence degree, determining the freeze ratio of the specified parameter, wherein the training convergence degree and the freeze ratio are positively correlated.

[0021] In one embodiment of this specification, during the training of the feature layer and inference layer, the training monitoring module continuously collects the training loss values ​​calculated on the validation set after each training epoch and stores these loss values ​​in chronological order in the training log database. When entering the parameter freeze decision stage, the convergence analysis submodule extracts the loss value sequence of the most recent consecutive training epochs from the training log database. It quantifies the loss fluctuation value by calculating the ratio of the sum of the absolute values ​​of the differences between adjacent epochs in this sequence to the initial loss value. The length of the loss value sequence is dynamically set according to the model complexity to ensure accurate reflection of training stability. The system's preset convergence threshold is a judgment standard pre-set based on the model architecture characteristics and training task type. When the calculated loss fluctuation value is less than this convergence threshold, the convergence judgment submodule marks the current training stage as having reached convergence. Simultaneously, it calculates the training convergence degree based on the relative magnitude of the loss fluctuation value and the convergence threshold. This convergence degree is negatively correlated with the loss fluctuation value; that is, the smaller the loss fluctuation, the higher the convergence degree. The parameter freezing control module determines the corresponding parameter freezing ratio based on the specific training convergence value and a preset mapping relationship. This mapping relationship ensures that the higher the training convergence, the larger the corresponding parameter freezing ratio. During the feature layer training phase, freezing objects are selected sequentially from the bottom parameters of the base layer according to the calculated freezing ratio, prioritizing the freezing of network parameters farther from the output layer. During the inference layer training phase, all parameters of the base layer and a portion of the parameters in the feature layer determined according to the freezing ratio are frozen simultaneously. Finally, the parameter freezing execution module removes the parameters marked as frozen from the list of parameters to be updated. These parameters only participate in forward propagation calculations during subsequent training, and their gradients are no longer involved in backpropagation updates, thus achieving adaptive parameter freezing management based on the dynamic convergence status of training.

[0022] Compared to traditional fixed parameter freezing schemes, this technical solution establishes a dynamic correlation mechanism between training convergence and the parameter freezing ratio, achieving intelligent management of the training process. It assesses the degree of training convergence by monitoring loss fluctuations in real time and dynamically adjusts the parameter freezing strategy accordingly. This ensures that the model can fully utilize the parameter update space for continued optimization when training is insufficient, while freezing underlying parameters in a timely manner to avoid redundant computation when training stabilizes. This dynamic adjustment mechanism significantly improves training efficiency and resource utilization. In the early stages of training, when model convergence is low, maintaining a larger number of trainable parameters helps to quickly capture data features. As training progresses and model convergence gradually improves, automatically increasing the parameter freezing ratio effectively reduces computational load while preventing damage to learned basic features. It can significantly accelerate the training process and reduce computational resource consumption while ensuring model performance, providing a more refined and intelligent parameter optimization strategy for training large-scale inference models.

[0023] The inference layer is trained using a pre-acquired scientific reasoning task corpus of scientific questions. Specifically, this includes: executing a joint optimization training process during training, which involves: determining the loss change rate and gradient norm corresponding to the current iteration to dynamically adjust the learning rate; calculating the main inference task loss and the auxiliary task loss to construct a multi-task regularization loss based on the main inference task loss, the auxiliary task loss, and a pre-defined regularization term; obtaining the sequence length and number of logical steps for each training sample in the scientific reasoning task corpus, where the sequence length is the number of tokens in the sample, and the number of logical steps is the number of inference steps contained in the sample; determining a sample complexity score based on the sequence length and number of logical steps for each training sample, and dynamically adjusting the training batch based on the sample complexity score.

[0024] The loss change rate and gradient norm corresponding to the current iteration are determined to dynamically adjust the learning rate. Specifically, this includes: obtaining the training loss value of the current iteration and the training loss value of the previous iteration from the training monitoring module, and calculating the loss change rate, where the loss change rate is the ratio of the absolute difference between the current iteration loss value and the previous iteration loss value to the previous iteration loss value; obtaining the gradient vector of the current iteration from the gradient calculation module, and calculating the gradient norm, where the gradient norm is the square root of the sum of the squares of the elements of the gradient vector; calculating the learning rate of the current iteration using the exponential decay formula based on the loss change rate and gradient norm, where the exponential decay formula includes an initial learning rate, a loss change rate adjustment coefficient, and a gradient norm adjustment coefficient; when the loss change rate exceeds a first predetermined threshold or the gradient norm exceeds a second predetermined threshold, an emergency decay coefficient is introduced into the exponential decay formula to reduce the learning rate.

[0025] In one embodiment of this specification, a dynamic learning rate adjustment process is initiated at the end of each iteration cycle during model training. First, the training loss value calculated for the current iteration is extracted from the training log of the current iteration. Simultaneously, the training loss value from the previous iteration is retrieved from the historical training records. Both loss values ​​originate from the output of the loss function calculated by the model during forward propagation on the training dataset. Based on these two consecutive iteration loss values, the rate of change of loss is calculated. Specifically, the absolute difference between the current iteration loss value and the previous iteration loss value is calculated, and then this absolute difference is divided by the previous iteration loss value to obtain a quantitative indicator of the relative change magnitude. This rate of change of loss reflects the fluctuation and convergence trend of the loss function during training. Simultaneously, the gradient vectors corresponding to all trainable parameters in the current iteration are obtained from the gradient calculation module. These gradient vectors are multidimensional data composed of the gradients of each parameter accumulated during backpropagation. By calculating the square root of the sum of the squares of each component of this gradient vector, the gradient norm is obtained. This gradient norm characterizes the overall magnitude and training stability of the current parameter update step size.

[0026] After obtaining the two key metrics, loss rate of change and gradient norm, the system calculates a new learning rate using an exponential decay formula. This formula, based on the initial learning rate, multiplies the loss rate of change and gradient norm by their respective adjustment coefficients, using these as negative inputs to the exponential part. A natural exponential function is used to smoothly decay the learning rate. The loss rate of change adjustment coefficient controls the impact of loss fluctuations on the learning rate, while the gradient norm adjustment coefficient controls the impact of gradient magnitudes on the learning rate. When the loss rate of change exceeds a first predetermined threshold, it indicates abnormal fluctuations in the loss function during training, suggesting that parameter updates may deviate from the optimal direction. Conversely, when the gradient norm exceeds a second predetermined threshold, it indicates that the gradient may experience explosive growth, posing a risk of training instability. In both cases, an emergency decay coefficient is introduced into the exponential decay formula. This coefficient acts as an additional multiplication factor to further reduce the learning rate, preventing drastic oscillations and training divergence during parameter updates. Finally, the dynamically adjusted learning rate is passed to the optimizer module for parameter update calculations in the next iteration, thus achieving fine-grained control over the training process.

[0027] Specifically, based on training dynamics, such as the loss change rate and gradient norm, the learning rate is adjusted in real time. This allows for rapid exploration of the parameter space in the early stages of training, stable updates in the middle stages, and fine-tuning in the later stages, avoiding oscillations and local optima. The formula for calculating the loss change rate ΔL is as follows: ,in Let be the training loss for the t-th iteration. This represents the loss from the previous iteration; a larger ΔL indicates greater fluctuation in parameter updates, requiring a reduction in the learning rate. Gradient norm. The calculation formula is: ,in Let be the gradient of the j-th parameter in the t-th iteration; A larger value indicates that the parameter update step size may be too large, requiring a reduction in the learning rate. Based on the loss rate of change and the gradient norm, the learning rate for the current iteration is calculated using the exponential decay formula, which includes the initial learning rate, a loss rate of change adjustment coefficient, and a gradient norm adjustment coefficient. The corresponding formula is expressed as follows , where the initial parameters γ is the emergency attenuation coefficient, which is applied when ΔL > 0.1 (excessive loss fluctuation) or When the risk of gradient explosion is high, γ = 0.8; otherwise, γ = 1.0. For example, in the 100th iteration, =0.8, =0.7, , ,but .

[0028] Compared to traditional fixed learning rate or simple step-by-step decay strategies, traditional learning rate scheduling methods often rely on preset schedules or fixed decay rules, failing to respond in real-time to dynamic changes during training. In the early stages of training, excessively high learning rates may cause parameter oscillations, while in the later stages, excessively low learning rates may lead to slow convergence. The technical solution in this specification, by simultaneously monitoring two key training metrics—the rate of change of loss and the gradient norm—can accurately capture the changing trends of the training state. It maintains an appropriate learning rate to accelerate convergence when training is stable and promptly reduces the learning rate to maintain stability when abnormalities occur. This dynamic adjustment mechanism is particularly suitable for complex training scenarios common in scientific research inference tasks, where different learning rates are required to adapt to varying optimization environments when dealing with inference problems of varying difficulty. By monitoring the rate of change of loss, the overall progress and convergence status of training can be perceived, avoiding blindly pushing forward training when the loss fluctuates drastically. By monitoring the gradient norm, the step size range of parameter updates can be controlled, preventing training instability caused by gradient anomalies. This provides a more comprehensive assessment of the training status, making learning rate adjustment more accurate and reliable. It can accelerate model convergence while ensuring training stability, improve training efficiency, and provide strong technical support for model optimization of complex inference tasks.

[0029] The process involves calculating the loss of the main inference task and the loss of the auxiliary task, and constructing a multi-task regularization loss based on these losses and a preset regularization term. Specifically, this includes: obtaining the model prediction result from the inference task output layer; calculating the main inference task loss based on the model prediction result and the ground truth labels, where the main inference task loss is the cross-entropy loss function; obtaining the auxiliary task prediction result from the auxiliary task output layer; calculating the auxiliary task loss based on the auxiliary task prediction result and the corresponding ground truth labels, where the auxiliary task includes textual entailment tasks, common sense reasoning tasks, or logical structure parsing tasks; obtaining model parameters from the model parameter storage module; calculating an L2 regularization term, where the L2 regularization term is the sum of squared model parameters; and weighting and summing the main inference task loss, the auxiliary task loss, and the L2 regularization term according to preset weight coefficients to obtain the multi-task regularization loss.

[0030] In one embodiment of this specification, after the forward propagation process of model training is completed, firstly, the model's prediction results for the current batch of samples are obtained from the inference task output layer. This output layer typically consists of a fully connected layer and a softmax function, and the output prediction probability distribution corresponds exactly to the format dimension of the true labels. Based on these prediction results, the cross-entropy loss function is used to calculate the loss of the main inference task. Specifically, the predicted probability distribution output by the model is compared with the true labels obtained from the training dataset, and the difference between the two distributions is calculated to quantify the model's performance on the main task. The true labels are derived from professionally labeled scientific research inference datasets, and each sample contains a verified standard answer. Simultaneously, prediction results for multiple auxiliary tasks are obtained in parallel from the auxiliary task output layer. These output layers have specific structural designs according to different auxiliary task types. For example, the textual entailment task uses a three-classification output structure, the common sense reasoning task uses a multi-option discriminant structure, and the logical structure parsing task uses a sequence labeling structure. Based on the prediction results of these auxiliary tasks, the loss for each auxiliary task is calculated. For the textual entailment task, the loss calculation involves comparing the model's output classification of entailment relationships with manually labeled semantic relationship tags. For the common sense reasoning task, the loss calculation involves matching the model's scores for candidate answers with the standard answers. For the logical structure parsing task, the loss calculation involves comparing the syntax tree structure generated by the model with the tree-like labels annotated by experts. Regarding model parameter regularization, the values ​​of all currently trainable parameters are obtained from the model parameter storage module. These parameters include the weight matrices and bias vectors of each Transformer layer. The L2 regularization term is obtained by calculating the sum of the squares of all parameter values. This regularization term constrains the magnitude of the parameter values, preventing overfitting caused by excessive reliance on certain specific parameters.

[0031] Finally, the main inference task loss, the losses of multiple auxiliary tasks, and the L2 regularization term are weighted and summed according to preset weight coefficients. The weight of the main task loss is usually set to the highest value to ensure the training effect of the main task. The weight coefficients of each auxiliary task loss are assigned different weight coefficients according to their relevance to the main task. The weight of the L2 regularization term is adjusted according to the model complexity and the amount of training data. A unified multi-task regularization loss function is constructed through this weighted summation method. This loss function considers the optimization objective of the main task, incorporates the constraints of the auxiliary tasks, and also includes a parameter complexity control mechanism, providing comprehensive optimization guidance for model training.

[0032] Specifically, based on the loss function of the main inference task, an auxiliary inference task loss and an L2 regularization term are introduced to constrain the parameter update magnitude, prevent model overfitting, and enhance generalization ability. Auxiliary tasks related to the main inference task are selected, including textual entailment tasks (such as SNLI, MNLI), commonsense reasoning tasks (such as CSQA, HellaSwag), and logical structure parsing tasks (such as consistency parsing); an improved loss function is then constructed. The formula is as follows: ,in Loss for the main inference task (such as cross-entropy loss). Let λ1 be the loss for the m-th auxiliary task, λ1 be the auxiliary task weight (set to 0.3~0.6), and λ2 be the L2 regularization weight (set to 1e-4~1e-3). ,in Let j be the j-th model parameter.

[0033] Traditional methods typically optimize only for a single primary reasoning task, neglecting the positive impact of related auxiliary tasks on model performance. This can easily lead to overfitting of the model to specific task data features, resulting in a significant performance decline when encountering new problem types or cross-domain applications. The technical solution in this specification introduces multiple auxiliary tasks closely related to the primary task, enabling the model to simultaneously acquire different levels of language understanding and reasoning abilities during the learning process. The textual entailment task enhances the model's depth of understanding of semantic relationships, the common sense reasoning task improves the model's ability to apply real-world knowledge, and the logical structure parsing task strengthens the model's ability to analyze language structures. The synergistic development of these abilities gives the model stronger adaptability and generalization capabilities when facing complex scientific reasoning tasks. Furthermore, the technical solution in this specification organically combines multi-task learning with regularization techniques. The additional supervision signals provided by auxiliary tasks and the constraints on the parameter space imposed by L2 regularization work together to prevent model overfitting. This multi-layered regularization mechanism not only effectively controls the complexity of the model, but also enables the model to learn more generalizable feature representations, avoiding excessive reliance on noise and specific patterns in the training data. It ensures the model's balanced performance across multiple tasks, as well as its simplicity and robustness. The resulting inference model maintains its performance on the main task while exhibiting stronger generalization ability and stability, making it better suited to the diverse inference needs and application scenarios in the scientific research field.

[0034] Based on the sequence length and number of logical steps of each training sample, a sample complexity score is determined, and the training batch is dynamically adjusted based on this score. Specifically, this includes: determining the maximum sequence length and maximum number of logical steps in the training samples based on the sequence length and number of logical steps of each training sample; determining the sequence length ratio by comparing the sequence length of each training sample with the maximum sequence length, and determining the step count ratio by comparing the number of logical steps of each training sample with the maximum number of logical steps; calculating the sample complexity score by weighted summing of the sequence length ratio and the step count ratio; dividing the training samples into multiple complexity levels based on the sample complexity score, and assigning a corresponding batch size to each complexity level, wherein the lower the sample complexity score, the larger the assigned batch size, and the batch size assigned to each complexity level is not less than a preset minimum batch size.

[0035] In one embodiment of this specification, the sequence length data of each training sample is obtained. This data comes from the token sequence length statistics obtained after the token segmenter processes the original text. Simultaneously, the number of logical steps is extracted from the sample annotation information. This data is the number of inference steps obtained after the dependency parser performs syntactic structure parsing on the sample text. Specifically, it is determined by identifying logical connectors, inference indicator words, and semantic dependency relations in the text to identify the independent inference steps contained in each sample. After obtaining the sequence length and number of logical steps of all training samples, the maximum sequence length and the maximum number of logical steps in the entire training dataset are determined by traversal comparison. These two maximum values ​​will serve as the benchmark reference values ​​for subsequent standardization calculations. For each training sample, the ratio of its sequence length to the maximum sequence length is calculated to obtain the standardized sequence length ratio. This ratio reflects the relative complexity of the current sample in the length dimension relative to the longest sample in the dataset. Simultaneously, the ratio of its number of logical steps to the maximum number of logical steps is calculated to obtain the standardized number of steps ratio. This ratio reflects the relative difficulty of the current sample in the inference complexity dimension relative to the most complex sample in the dataset. The sample complexity score is calculated by weighting and summing these two standardized ratios. The weight coefficient of the sequence length ratio reflects the influence of sequence length on training difficulty, while the weight coefficient of the logical steps ratio reflects the influence of inference complexity on training difficulty. The setting of these two weight coefficients is based on prior analysis of the training characteristics of different types of samples.

[0036] After obtaining the complexity scores of all samples, the training samples are divided into multiple complexity levels according to a preset complexity level classification threshold. Each level corresponds to different training characteristics: low complexity level includes samples with short sequences and simple inference steps; medium complexity level includes samples with moderate length and inference difficulty; and high complexity level includes samples with long sequences and complex inference steps. A corresponding batch size is assigned to each complexity level. Low complexity level, due to its higher training stability, is assigned a larger batch size to improve training efficiency; medium complexity level is assigned a moderate batch size to balance efficiency and stability; and high complexity level is assigned a smaller batch size to ensure the stability of the training process. For example, the batch size B is adjusted according to S. When S∈[0,0.3], ( (Set to 64~128); when S∈(0.3,0.7], When S∈(0.7,1], Furthermore, B should not be less than 8 after each adjustment to avoid unstable training due to excessively small batch sizes.

[0037] When allocating batch sizes, a minimum batch size constraint is enforced to ensure that the batch size for each level is not lower than this preset minimum value. This minimum batch size is determined based on theoretical analysis and experimental verification of gradient estimation stability, thus avoiding the problem of excessively large gradient estimation variance caused by excessively small batches. During training, the data loader loads data in groups according to the complexity level of the samples, ensuring that the samples within each training batch have similar complexity characteristics, thereby improving training efficiency while maintaining the stability of the training process.

[0038] Specifically, the batch size is dynamically adjusted based on sample complexity. Complex samples use smaller batches to avoid gradient oscillations, while simple samples use larger batches to improve efficiency, balancing training stability and speed. The formula for calculating the sample complexity score S is as follows: Where n is the length of the sample sequence, such as the number of tokens in the stem and options of a math problem; The maximum sequence length in the training set is assumed to be 2048; s is the number of logical steps in the sample, such as the number of steps in solving a math problem. It is statistically analyzed through dependency parsing, such as "calculate the part inside the parentheses first → then calculate the multiplication → finally calculate the addition" is 3 steps; The maximum number of logical steps in the training set is assumed to be 10. For example, with sample n=1024 and s=5, S=0.5×(1024 / 2048)+0.5×(5 / 10)=0.5×0.5+0.5×0.5=0.5. The preset batch size adjustment rule is: maximum batch size B_max=128 (based on 32GB memory on a single GPU). When S∈[0,0.3], it represents simple samples, such as short-sequence single-step inference with B=128; when S∈(0.3,0.7], it represents medium-complexity samples, such as medium-sequence 3-5-step inference with B=80; when S∈(0.7,1], it represents complex samples, such as long-sequence 8-10-step inference with B=48. The constraint B≥8 is used to avoid inaccurate gradient estimation due to excessively small batch sizes.

[0039] Traditional fixed-batch methods apply a uniform batch size to all samples, which cannot adapt to the different training characteristics of samples with varying complexity. When processing simple samples, the batch size limitation prevents full utilization of parallel computing efficiency, while for complex samples, excessively large batches may lead to gradient anomalies and training instability. The technical solution in this specification comprehensively considers the complexity characteristics of samples from two dimensions: sequence length and the number of logical steps. This allows for accurate identification of differences in training difficulty among different samples, and the implementation of differentiated batch size strategies accordingly. This fine-grained batch adjustment mechanism makes the model training process more intelligent. Using large batches for simple samples significantly improves data throughput and computational efficiency, while using small batches for complex samples effectively controls gradient variation and training fluctuations. Compared to single-dimensional adjustment methods that only consider sequence length or logical complexity, the dual considerations of the technical solution in this specification provide a more comprehensive assessment of the actual training difficulty of samples, avoiding unreasonable batch allocation due to incomplete evaluation dimensions. By ensuring the constraint of the minimum batch value, the technical solution in this specification improves training efficiency while maintaining the stability of the training process, avoiding gradient estimation bias caused by extremely small batches. This dynamic batch adjustment strategy is suitable for mixed-complexity datasets commonly used in scientific reasoning tasks, and can significantly accelerate model convergence while ensuring training quality.

[0040] The input text sequence is processed by a pre-loaded, pre-trained inference model. For input text sequences whose length exceeds a predetermined threshold, long sequence attention optimization is performed. Specifically, this includes: dividing the input text sequence into multiple windows of a predetermined fixed window size; for each window, calculating the importance score of each token based on the mean of the self-attention weights, and identifying tokens with high importance scores as key nodes; calculating the attention weights among multiple key nodes to form a sparse attention matrix; and weighting and summing the window attention weight matrix and the sparse attention weight matrix according to a predetermined fusion coefficient to generate the final attention weight matrix for long sequence attention optimization.

[0041] In one embodiment of this specification, after the model receives the preprocessed input text sequence, it first detects the sequence length. When the sequence length exceeds a preset threshold, the long sequence attention optimization process is automatically initiated. The complete input text sequence is divided into multiple consecutive windows according to a preset fixed window size. This fixed window size is a standard value set based on a balance between hardware memory capacity and computational efficiency. Each window contains a continuous and non-overlapping subsequence of tokens. If the last window is not long enough, padding is used to maintain a uniform size. After window division, standard self-attention calculation is performed independently on each window to generate an attention weight matrix between tokens within the window. This matrix fully preserves the local semantic association information within the window. Subsequently, the importance score of each token is calculated based on the average attention weight values ​​of each token with all other tokens in the self-attention calculation. This score reflects the semantic coreness of the current token within the window. All tokens within the window are sorted in descending order of importance score, and a certain proportion of the top-ranked tokens are selected as key nodes. These key nodes are usually core words that bear the main semantic functions and logical relationships. After obtaining the key node set for all windows, these key nodes from different windows are reorganized into a new sequence, and the attention weights between each node in this key node sequence are calculated to form a sparse attention matrix. This matrix captures the global semantic relationships across different windows. Finally, the window attention weight matrix is... With sparse attention weight matrix (K is the total number of key nodes) The weighted sum is calculated according to the preset fusion coefficients to generate the final attention weight matrix. The fusion coefficient of the window attention weight matrix determines the degree of preservation of local semantic information, while the fusion coefficient of the sparse attention weight matrix controls the integration strength of global key information. This fusion mechanism realizes the organic combination of detailed local semantic analysis and global key information capture, which significantly reduces the computational complexity while maintaining the model's ability to understand long text sequences.

[0042] Specifically, by combining window attention (local association) and sparse attention (global key association), the computational complexity of attention is reduced, enabling efficient processing of long sequences (n>2048). The window attention calculation process is as follows: First, the long sequence (e.g., n=4096) is divided into 8 windows of size W=512; for each window, the full attention weights between tokens are calculated. This process captures local semantic relationships. For each window, an importance score for the token is calculated (based on the mean of the self-attention weights). The top-15% of tokens are selected as key nodes, with approximately 77 key nodes per window, totaling 616. Attention weights are then calculated among all key nodes. It captures key semantic relationships globally while ignoring redundant computations between non-critical nodes. This is done according to the final attention weight matrix. , window attention weight matrix With sparse attention weight matrix The final attention weight matrix is ​​generated by weighting and summing the values ​​according to a preset fusion coefficient. The fusion coefficient of the window attention weight matrix is ​​0.8, and the fusion coefficient of the sparse attention weight matrix is ​​0.2, prioritizing local semantic coherence while preserving key global associations. When n=4096, the complexity of traditional full attention is... The optimized complexity is 10,000. The complexity is reduced by more than 85%.

[0043] Traditional methods require calculating attention weights between all tokens when processing long sequences, leading to a quadratic increase in computational complexity and memory usage with sequence length, severely limiting the model's ability to process long text. The technical solution in this specification reduces computational complexity from quadratic to linear by dividing long sequences into fixed-size windows and performing full attention computation within each window. This allows the model to efficiently process text sequences far exceeding the length limitations of traditional methods. Furthermore, the technical solution in this specification balances computational efficiency with semantic preservation through key node selection and a sparse attention mechanism. While simple window attention is computationally efficient, it completely loses the semantic connections between windows, resulting in a decreased ability of the model to understand long-range dependencies. The technical solution in this specification, by selecting key nodes in each window and establishing sparse connections across windows, preserves important global semantic information while avoiding the huge overhead of full attention computation. Compared to other existing sparse attention methods, the key node selection in the technical solution of this specification, based on the mean attention weight—an indicator with clear semantic meaning—can more accurately identify tokens that play a key role in semantic expression, thus achieving better performance preservation at the same sparsity. This optimization strategy is particularly suitable for long document processing scenarios common in scientific research reasoning tasks, enabling the model to process complete scientific literature, technical reports and other long text materials with limited computing resources.

[0044] Step S103: Based on the final attention output, scientific research reasoning results are generated through the reasoning layer of the pre-trained large-scale inference model.

[0045] The results of this scientific research reasoning include textual implications, common sense reasoning answers, or logical structure analysis outputs.

[0046] In one embodiment of this specification, after long-sequence attention optimization is completed and the final attention output is obtained, semantic representations processed by the preceding layers are received. These representations already contain optimized local and global semantic information. The inference attention layer, as the core component of the inference layer, first performs multi-round transformation processing on the input semantic representations. This layer employs a specially designed inference-oriented attention mechanism, identifying inference elements such as causal relationships, conditional relationships, and evidence-supporting relationships in the text by calculating the logical association strength between different semantic units. Specifically, this layer constructs a logical relationship graph, where nodes represent important semantic concepts and edges represent different types of logical relationships. Through multiple iterations of information transmission and updates, the weights of key inference paths are gradually strengthened. After completing the logical relationship modeling, the output prediction layer begins operation. This layer contains multiple parallel output heads, each responsible for different types of scientific research inference tasks.

[0047] For the text entailment judgment task, the system adopts a three-class output structure. Based on the semantic representation output by the inference layer, it calculates the probability distribution of the logical relationship between two text fragments. By comparing the semantic consistency, logical coherence, and evidence support between the premise text and the hypothesis text, it finally outputs a three-class judgment result of entailment, contradiction, or neutrality. Among them, entailment means that the premise text must lead to the hypothesis text, contradiction means that there is a logical conflict between the premise text and the hypothesis text, and neutrality means that the premise text cannot determine the truth or falsehood of the hypothesis text.

[0048] For the common sense reasoning answer generation task, a multi-option discrimination mechanism is adopted. First, the core query intent is extracted from the question. Then, the credibility of each candidate answer is evaluated. The evaluation process comprehensively considers the semantic matching degree between the answer and the question, the common sense rationality of the answer itself, and the consistency between the answer and the context. Finally, the answer with the highest comprehensive score is selected as the output result. During the processing, the system will call the built-in common sense knowledge base to verify the rationality of the answer.

[0049] For the logical structure parsing task, a method combining sequence labeling and tree structure generation is adopted. First, logical connectors, inference indicators and propositional boundaries in the text are identified through sequence labeling. Then, a logical structure tree is constructed based on these labeling results. This tree structure clearly shows the argumentation framework, inference chain and evidence relationship in the text, and finally outputs a standardized logical structure representation.

[0050] Before generating the final result, a consistency check is performed on the results from each output head to ensure there are no logical contradictions between the outputs of different tasks. Simultaneously, a confidence assessment mechanism filters out results with low credibility. Finally, the generated scientific reasoning results are packaged in a standardized format, including a task type identifier, the main body of the reasoning result, a confidence score, and supplementary explanation information. This information is then transmitted to downstream application systems or directly returned to the user through the result output interface. The entire processing fully considers the characteristics of scientific reasoning tasks, ensuring the accuracy and interpretability of the reasoning results while maintaining processing efficiency.

[0051] This solution achieves collaborative processing of various scientific reasoning tasks through a shared inference layer architecture and unified result generation. This allows the model to simultaneously utilize supervision signals from different tasks, gaining stronger reasoning capabilities based on parameter sharing and knowledge transfer. For textual implication judgment tasks, it not only considers surface semantic similarity but also delves into the logical derivation relationships between texts through a dedicated inference attention mechanism, accurately identifying complex situations with different semantic expressions but logical implications. In commonsense reasoning tasks, a multi-dimensional evaluation mechanism comprehensively considers various aspects of the answer, avoiding the one-sided judgment problem that easily occurs in traditional methods. It exhibits stronger robustness, especially when dealing with complex commonsense problems requiring multi-step reasoning.

[0052] The technical solutions implemented in this specification construct an end-to-end processing flow for scientific reasoning tasks. By introducing a long-sequence attention optimization mechanism and employing a hybrid computing mode that integrates local window attention and global sparse attention, a precise balance is achieved between computational complexity and semantic preservation. This allows the model to maintain sensitivity to local details while retaining an understanding of the global logical structure when processing long texts such as scientific literature and experimental reports, breaking through the structural limitations of traditional models in long-sequence processing at the principle level. The reasoning layer adopts a reasoning-oriented attention mechanism, explicitly modeling reasoning elements such as causal relationships, evidence support relationships, and conditional dependencies in the text through the construction of logical relationship graphs and multi-round iterative information transmission. This enables the model to go beyond simple word co-occurrence statistics and deeply understand the implicit logical context and argumentation structure in scientific texts. Traditional methods often require the deployment of independent model systems for different reasoning tasks, leading to redundant computing resources, increased maintenance costs, and an inability to fully utilize the correlation between tasks. The technical solutions implemented in this specification, through a shared underlying encoder and a dedicated multi-task reasoning layer, achieve positive knowledge transfer between different reasoning tasks while maintaining the efficiency of model parameters. By employing a pre-trained model loading mechanism and dynamic optimization strategies, the system's practicality and stability in real-world deployment environments are ensured. Parameter sharing within the pre-trained model significantly reduces computational overhead during inference, while the dynamic optimization mechanism for long sequence processing enables the system to adaptively adjust its computational strategy based on input features, maximizing resource utilization efficiency while maintaining result quality. This design allows the technical solution to meet the needs of common research applications such as large-scale literature analysis and complex problem-solving.

[0053] This specification also provides a large-scale inference model training system for scientific research inference tasks. The system includes: a hierarchical pre-training module, used to divide the large-scale inference model into a basic layer, a feature layer, and an inference layer according to function, and to implement pre-training in stages while gradually freezing low-level parameters to reduce computational resource consumption during training; an adaptive learning rate scheduling module, used to dynamically adjust the learning rate based on the loss change rate and gradient norm during training to adapt to the parameter update needs of the model at different training stages and accelerate convergence; a multi-task regularization module, used to introduce auxiliary inference task loss and L2 regularization terms on the basis of the main inference task loss function to constrain the parameter update amplitude, prevent model overfitting, and enhance generalization ability; a long sequence attention optimization module, used to combine sliding window attention and sparse attention mechanisms to reduce the attention computation complexity of the large-scale inference model when processing long sequences and reduce memory usage; and a dynamic batch adjustment module, used to calculate the sample complexity score based on the inference complexity of the training samples (including sequence length and number of logical steps), adaptively adjust the training batch size, and balance training efficiency and stability.

[0054] The hierarchical pre-training module includes a hierarchical partitioning submodule, a phased training submodule, and a parameter freezing submodule. The hierarchical partitioning submodule is used to divide the large inference model into three layers: the base layer (responsible for basic language modeling, including word embedding layer and basic Transformer encoder), the feature layer (responsible for semantic feature extraction, including advanced Transformer encoder and feature fusion layer), and the inference layer (responsible for logical inference, including inference attention layer and output prediction layer). The phased training submodule is used to conduct training in the order of "base layer → feature layer → inference layer": The base layer uses general corpora (such as WikiText-103, BookCorpus) to train language modeling tasks, with 10-15 training rounds; the feature layer uses domain inference corpora (such as scientific literature abstracts, legal provisions) to train semantic matching tasks, with 5-8 training rounds, freezing 70%-80% of the underlying parameters of the base layer during training; the inference layer uses specialized inference task corpora (such as GSM8K mathematical reasoning, RACE reading comprehension) to train, with 8-12 training rounds, freezing all parameters of the base layer and 50%-60% of the parameters of the feature layer during training. The parameter freezing submodule is used to determine the proportion of parameters to be frozen during high-level training based on the training convergence of the previous level (a loss fluctuation of less than 0.001 is considered convergence); the higher the convergence, the larger the freezing proportion.

[0055] The adaptive learning rate scheduling module includes a parameter freezing submodule and a learning rate update submodule; a dynamic factor calculation submodule is used to calculate the rate of change of loss in the current iteration. (where L_t is the loss value in the t-th iteration) and gradient norm (in (The gradient of the j-th parameter in the t-th iteration); the learning rate update submodule is used to update the learning rate according to the formula... Adjust the learning rate, where α_0 is the initial learning rate (set to 1e-5~3e-5), and k1 and k2 are adjustment coefficients (k1∈[0.2,0.5], k2∈[0.1,0.3]). When ΔL>0.1 or When the value is greater than 5, an additional attenuation factor of 0.8 is applied to prevent parameter oscillation.

[0056] The multi-task regularization module includes a multi-task loss calculation submodule. The auxiliary task selection submodule is used to select auxiliary tasks related to the main inference task, including textual entailment tasks (such as SNLI, MNLI), commonsense reasoning tasks (such as CSQA, HellaSwag), and logical structure parsing tasks (such as consistency parsing). The multi-task loss calculation submodule is used to construct the improved loss function. ,in Loss for the main inference task (such as cross-entropy loss). Let λ1 be the loss of the m-th auxiliary task, λ1 be the auxiliary task weight (set to 0.3~0.6), and λ2 be the L2 regularization weight (set to 1e-4~1e-3). (in (This refers to the j-th parameter of the model).

[0057] The long sequence attention optimization module includes a window attention submodule, a sparse attention submodule, and an attention fusion submodule. The window attention submodule divides the long sequence (length > 1024) into segments of a fixed window size W (W is set to 256~512), and performs full attention computation within each window. The attention weight matrix within the window is as follows: The sparse attention submodule is used to establish sparse connections between windows. For each window's key nodes (selected based on the top-10% to 20% of token importance scores), it calculates the attention weights of the key nodes with other windows, forming a sparse attention matrix. (Where K is the total number of key nodes). The attention fusion submodule is used to fuse window attention and sparse attention: (Where α is the fusion coefficient, set to 0.7~0.9), reducing the computational complexity of attention from... Reduce to (where n is the sequence length).

[0058] The dynamic batch adjustment module includes a sample complexity scoring submodule and a batch size adjustment submodule. The sample complexity scoring submodule is used to calculate sample complexity. Where n is the sample sequence length, The maximum sequence length in the training set is s, where s is the number of logical reasoning steps contained in the sample (statistically determined through dependency parsing). To determine the maximum number of logical steps in the training set, w1 and w2 are weights (w1 = 0.4~0.6, w2 = 0.4~0.6). The batch size adjustment submodule is used to adjust the batch size B according to S: when S∈[0,0.3], (B_max is set to 64~128); when S∈(0.3,0.7], When S∈(0.7,1], Furthermore, B should not be less than 8 after each adjustment to avoid unstable training due to excessively small batch sizes.

[0059] The inference large model training system also includes a training monitoring module, which is used to monitor the training metrics of the model in real time (including training loss, validation accuracy, gradient norm, and parameter update magnitude). When the validation accuracy does not improve for 5 consecutive rounds or the training loss increases for 3 consecutive rounds, the system triggers parameter rollback (rolling back to the optimal parameters of the previous 3 rounds) and readjusts the learning rate and batch size.

[0060] This specification also provides an embodiment of a scientific research reasoning task processing device, such as... Figure 2 As shown, the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described method.

[0061] This specification also provides a non-volatile computer storage medium storing computer-executable instructions configured to perform the above-described method.

[0062] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0063] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for processing scientific research reasoning tasks, characterized in that, The method includes: Receive a scientific research reasoning task request, wherein the scientific research reasoning task request includes an input text sequence, the input text sequence being from a scientific literature database or a scientific research query submitted by a user; The input text sequence is processed by a pre-loaded, pre-trained large inference model. For input text sequences whose sequence length exceeds a predetermined threshold, long sequence attention optimization is performed to generate the final attention output. Based on the final attention output, scientific research reasoning results are generated through the reasoning layer of the pre-trained reasoning model, wherein the scientific research reasoning results include textual implication judgments, common sense reasoning answers, or logical structure parsing outputs.

2. The scientific research reasoning task processing method according to claim 1, characterized in that, Before processing the input text sequence using a pre-loaded, pre-trained large inference model, the method further includes: A base layer is trained on a general corpus of a pre-acquired public text dataset to perform a language modeling task, wherein the base layer includes a word embedding layer and a base Transformer encoder. Based on a pre-acquired scientific literature abstract database, a domain-specific scientific research corpus dataset is obtained to train a feature layer using the domain-specific scientific research corpus dataset to perform a semantic matching task. During the training of the feature layer, specified parameters of the base layer are frozen, wherein the proportion of parameters frozen is determined based on the training convergence of the base layer; The inference layer is trained using a pre-acquired corpus of scientific reasoning tasks based on a dataset of scientific questions to perform inference prediction tasks. During the training of the inference layer, all parameters of the base layer and specified parameters of the feature layer are frozen.

3. The scientific research reasoning task processing method according to claim 2, characterized in that, The method further includes: During the training of the feature layer and the training of the inference layer, the change range of the training loss value corresponding to multiple training sessions is obtained to determine the loss fluctuation value. When the loss fluctuation value is determined to be less than the preset convergence threshold, it is determined to be converged, so as to determine the degree of training convergence. The freezing ratio of the specified parameter is determined based on the training convergence, wherein the training convergence and the freezing ratio are positively correlated.

4. The scientific research reasoning task processing method according to claim 2, characterized in that, The inference layer is trained using a pre-acquired corpus of scientific reasoning tasks based on scientific question datasets, specifically including: During the training of the inference layer, a joint optimization training process is executed, which includes: Determine the rate of change of loss and gradient norm corresponding to the current iteration in order to dynamically adjust the learning rate; Calculate the main inference task loss and the auxiliary task loss, and construct a multi-task regularization loss based on the main inference task loss, the auxiliary task loss and the preset regularization term; Obtain the sequence length and number of logical steps for each training sample in the scientific research reasoning task corpus, wherein the sequence length is the number of tokens in the sample, and the number of logical steps is the number of reasoning steps contained in the sample; Based on the sequence length and number of logical steps of each training sample, a sample complexity score is determined, and the training batch is dynamically adjusted based on the sample complexity score.

5. The scientific research reasoning task processing method according to claim 4, characterized in that, Determine the rate of change of loss and gradient norm for the current iteration to dynamically adjust the learning rate, specifically including: The training loss value of the current iteration and the training loss value of the previous iteration are obtained from the training monitoring module, and the loss change rate is calculated, wherein the loss change rate is the ratio of the absolute difference between the current iteration loss value and the previous iteration loss value to the previous iteration loss value. Obtain the gradient vector of the current iteration from the gradient calculation module, and calculate the gradient norm, wherein the gradient norm is the square root of the sum of the squares of the elements of the gradient vector; Based on the loss rate of change and the gradient norm, the learning rate for the current iteration is calculated using the exponential decay formula, which includes the initial learning rate, the loss rate of change adjustment coefficient, and the gradient norm adjustment coefficient. When the rate of change of loss exceeds a first predetermined threshold or the gradient norm exceeds a second predetermined threshold, an emergency decay coefficient is introduced into the exponential decay formula to reduce the learning rate.

6. The scientific research reasoning task processing method according to claim 4, characterized in that, Calculate the main inference task loss and the auxiliary task loss, and construct a multi-task regularization loss based on the main inference task loss, the auxiliary task loss, and a preset regularization term, specifically including: The model prediction results are obtained from the output layer of the inference task. The loss of the main inference task is calculated based on the model prediction results and the true labels. The loss of the main inference task is the cross-entropy loss function. The auxiliary task prediction results are obtained from the auxiliary task output layer. The auxiliary task loss is calculated based on the auxiliary task prediction results and the corresponding real labels. The auxiliary task includes textual entailment task, common sense reasoning task or logical structure parsing task. Model parameters are obtained from the model parameter storage module, and the L2 regularization term is calculated, wherein the L2 regularization term is the sum of squares of the model parameters; The loss of the main inference task, the loss of the auxiliary task, and the L2 regularization term are weighted and summed according to preset weight coefficients to obtain the multi-task regularization loss.

7. The scientific research reasoning task processing method according to claim 4, characterized in that, Based on the sequence length and number of logical steps of each training sample, a sample complexity score is determined, and the training batch is dynamically adjusted based on the sample complexity score, specifically including: Based on the sequence length and number of logical steps of each training sample, determine the maximum sequence length and the maximum number of logical steps in the training sample; The sequence length ratio is determined by the ratio of the sequence length of each training sample to the maximum sequence length, and the number of steps ratio is determined by the ratio of the number of logical steps of each training sample to the maximum number of logical steps. The sample complexity score is calculated by weighted summation of the sequence length ratio and the number of steps ratio. The training samples are divided into multiple complexity levels based on the sample complexity score, and a corresponding batch size is assigned to each complexity level. The lower the sample complexity score, the larger the assigned batch size. The batch size assigned to each complexity level is not less than a preset minimum batch value.

8. The scientific research reasoning task processing method according to claim 1, characterized in that, The input text sequence is processed by a pre-loaded, pre-trained large inference model. For input text sequences whose length exceeds a predetermined threshold, long sequence attention optimization is performed, specifically including: The input text sequence is divided into multiple windows according to a preset fixed window size, wherein the fixed window size is a preset value; For each window, an importance score for each token is calculated based on the mean of the self-attention weights, and tokens with high importance scores are identified as key nodes. Calculate the attention weights among multiple key nodes to form a sparse attention matrix. Then, sum the window attention weight matrix and the sparse attention weight matrix by weighting them according to a preset fusion coefficient to generate the final attention weight matrix for long sequence attention optimization.

9. A scientific research reasoning task processing device, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-8.

10. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are configured to perform the method as described in any one of claims 1-8.

Images