Method for Quantifying the Value of Scientific Problems Embedded with a Multi-Dimensional Mixture-of-Experts Mechanism

Abstract

The present application provides a method for quantifying the value of scientific questions by embedding a multi-dimensional mixture-of-experts mechanism, which relates to the technical field of data processing and includes: reading predetermined quantification dimensions; introducing a predetermined triple strategy to construct a target instruction fine-tuning dataset; performing parameter quantification based on the fine-tuning principle of the QLoRA large language model; obtaining a scientific question text, performing semantic feature analysis through a gating network, and obtaining the target weight coefficient of the scientific question text according to the text semantics; analyzing the scientific question text through a target recognition large model and combining the target weight coefficient to obtain a target quantification result. Through the present application, the technical problem in the prior art that due to relying on expert evaluation and subjective judgment, there are inconsistent evaluation criteria, resulting in low efficiency of scientific question value evaluation can be solved. By integrating multiple dimensions and combining large language models for fine-tuning, the internal evaluation of the value of scientific questions is realized, and the efficiency of scientific question value evaluation is improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a method for quantifying the value of scientific problems by embedding a multi-dimensional hybrid expert mechanism. Background Art

[0002] Scientific problems mainly focus on the phenomena and laws in the field of natural science, and usually involve the verification or extension of hypotheses, theories, and models. The quantification of the value of scientific problems refers to the process of quantitatively evaluating the value of scientific problems through the evaluation and estimation of certain standards and criteria. The existing methods for evaluating the value of scientific problems mainly rely on expert review and quantitative indicators. Expert review is one of the commonly used methods in the current evaluation of the value of scientific problems. It relies on domain experts to give subjective evaluations based on their own knowledge backgrounds and experiences, combined with their understanding of scientific problems. Although it can provide profound insights and rich background knowledge, there may be inconsistencies in the evaluation criteria due to the different knowledge structures and experiences of different experts, and there is often a lack of unity and standardization, which in turn affects the objectivity and repeatability of the evaluation. External indicators such as citation counts and journal impact factors are also widely used to measure the academic influence of scientific articles, journals, or scientific problems. Citation counts reflect the attention of a certain scientific problem in the academic community, and the journal impact factor measures the influence of a journal in the academic circle, which is usually regarded as an indirect standard for measuring the influence of scientific research problems. However, it has certain limitations in evaluating the value of scientific problems and cannot accurately reflect the actual innovation, frontier nature, and application potential of scientific problems.

[0003] In summary, there is a technical problem in the prior art that due to relying on expert evaluation and subjective judgment, there are inconsistent evaluation criteria, resulting in a low efficiency of evaluating the value of scientific problems. Summary of the Invention

[0004] The purpose of this application is to provide a method for quantifying the value of scientific problems by embedding a multi-dimensional hybrid expert mechanism, so as to solve the technical problem in the prior art that due to relying on expert evaluation and subjective judgment, there are inconsistent evaluation criteria, resulting in a low efficiency of evaluating the value of scientific problems.

[0005] In view of the above problems, the present application provides a method for quantifying the value of scientific questions by embedding a multi-dimensional mixture-of-experts mechanism. The method for quantifying the value of scientific questions by embedding a multi-dimensional mixture-of-experts mechanism includes: reading a predetermined quantization dimension, where the predetermined quantization dimension includes a target quantization dimension; introducing a predetermined triple strategy to construct a target instruction fine-tuning data set for the target quantization dimension; performing parameter quantization on the target instruction fine-tuning data set based on the fine-tuning principle of the QLoRA large language model to obtain a target recognition large model; obtaining a scientific question text, performing semantic feature analysis on the scientific question text through a gating network to obtain text semantics, and obtaining a target weight coefficient of the scientific question text on the target quantization dimension according to the text semantics; analyzing the scientific question text through the target recognition large model, and combining the target weight coefficient to obtain a target quantization result.

[0006] The technical solution provided in the present application has at least the following technical effects or advantages:

[0007] By reading a predetermined quantization dimension, where the predetermined quantization dimension includes a target quantization dimension; introducing a predetermined triple strategy to construct a target instruction fine-tuning data set for the target quantization dimension; performing parameter quantization on the target instruction fine-tuning data set based on the fine-tuning principle of the QLoRA large language model to obtain a target recognition large model; obtaining a scientific question text, performing semantic feature analysis on the scientific question text through a gating network to obtain text semantics, and obtaining a target weight coefficient of the scientific question text on the target quantization dimension according to the text semantics; analyzing the scientific question text through the target recognition large model, and combining the target weight coefficient to obtain a target quantization result. That is to say, through clear predetermined quantization dimensions, including multiple aspects such as theoretical value, application value, frontier value, and innovation value, using the triple strategy to construct a dedicated data set, using the fine-tuning principle of the QLoRA large language model for parameter quantization, introducing a gating network to automatically analyze the key information of the text, determining the weight coefficients of the scientific question text on each target quantization dimension, and obtaining a quantified evaluation result, which improves the efficiency of quantifying and evaluating the value of scientific questions.

[0008] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the following specifically illustrates the embodiments of the present application. It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. Brief Description of the Drawings

[0009] To more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0010] Figure 1 It is a schematic flowchart of a scientific problem value quantification method that embeds a multi-dimensional hybrid expert mechanism in the present application;

[0011] Figure 2 It is a schematic flowchart of constructing a target instruction fine-tuning dataset in the scientific problem value quantification method that embeds a multi-dimensional hybrid expert mechanism in the present application. Specific embodiments

[0012] By providing a scientific problem value quantification method that embeds a multi-dimensional hybrid expert mechanism, the present application solves the technical problem in the prior art that due to relying on expert evaluation and subjective judgment, there are inconsistent evaluation criteria, resulting in low efficiency of scientific problem value evaluation. Through clear predetermined quantification dimensions, including multiple aspects such as theoretical value, application value, frontier value, and innovation value, a dedicated dataset is constructed using a triple strategy, parameter quantification is carried out using the QLoRA large language model fine-tuning principle, a gating network is introduced to automatically analyze the key information of the text, the weight coefficients of the scientific problem text in each target quantification dimension are determined, and a quantified evaluation result is obtained, improving the efficiency of scientific problem value quantification evaluation.

[0013] Next, the technical solutions in the present application will be clearly and completely described with reference to the drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the example embodiments described here. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application. Additionally, it should be noted that for the convenience of description, only the parts related to the present application are shown in the drawings, rather than all of them.

[0014] Embodiment, please refer to the attached Figure 1 The present application provides a scientific problem value quantification method that embeds a multi-dimensional hybrid expert mechanism. Among them, the scientific problem value quantification method that embeds a multi-dimensional hybrid expert mechanism specifically includes the following steps:

[0015] Step 1: Read the predetermined quantification dimensions, and the predetermined quantification dimensions include target quantification dimensions.

[0016] The predetermined quantification dimensions at least include a theoretical value dimension, an application value dimension, a frontier value dimension, and an innovation value dimension.

[0017] Specifically, the predetermined quantification dimension is a pre-set evaluation index used to measure the value of scientific problems. Each dimension represents an aspect of the value of scientific problems, such as theoretical value, application value, frontier value, and innovation value, etc. The quantification dimension is essentially a standardized and numerical index of the evaluation object (such as scientific problems, technical papers, etc.) in certain specific aspects, used to measure its different values or characteristics. The target quantification dimension is a subset of the predetermined quantification dimension, which is a set of dimensions set for a specific evaluation target (such as the value evaluation of scientific problems), and is set to solve specific problems and guide how to evaluate the target.

[0018] The predetermined quantification dimension at least includes the theoretical value dimension, application value dimension, frontier value dimension, and innovation value dimension. The theoretical value dimension measures the contribution of scientific problems or research results to the theoretical field. Its evaluation criteria include developing new theories or theoretical frameworks (that is, creating a brand-new theoretical system to explain certain phenomena or fields), improving or expanding existing theoretical systems (that is, revising and perfecting existing theories to make them more complete or applicable), discovering new scientific laws or principles (that is, researching and revealing previously unknown natural laws or scientific principles), establishing new mathematical models (that is, the mathematical structure describing or predicting phenomena), providing theoretical explanations for observed phenomena (that is, using theories to explain the data or phenomena obtained in experiments or observations), etc.

[0019] The application value dimension measures the application potential of scientific problems or research results in practice. Its evaluation criteria include solving practical industrial or technical problems (that is, technologies or theories can solve specific problems in actual applications), demonstrating commercial potential or economic benefits (that is, the economic benefits that technologies or products can bring in the market), improving existing technical processes or methods (that is, optimizing existing technical processes or methods to improve efficiency or effectiveness), demonstrating the feasibility of real-world applications (that is, whether technologies or theories can be successfully applied in the actual environment), providing cost-effective or efficient solutions (that is, the proposed solutions are superior to existing solutions in terms of cost and efficiency), etc.

[0020] The frontier value dimension measures the role of scientific problems or research results in promoting the development of the discipline frontier. Its evaluation criteria include leading or opening up new research directions (that is, research work can guide the scientific community to explore new fields), promoting frontier technologies (that is, promoting technologies to be at the forefront of current scientific or engineering practices), facilitating interdisciplinary breakthroughs (that is, research can cross different disciplinary boundaries to generate new insights or technologies), answering unsolved scientific problems (that is, research can solve long-standing scientific problems), creating new research paradigms (that is, research can establish brand-new research models or frameworks), etc.

[0021] The innovation value dimension measures the innovation of scientific problems or research results, evaluating whether a study has put forward original scientific discoveries, innovative methodologies, technologies or tools, or whether it has made unique innovations in existing methods. Its evaluation criteria include making original scientific discoveries (i.e., the study can bring unprecedented scientific knowledge or understanding), developing new methodologies or technologies (i.e., creating brand-new technologies or research methods), creating new tools or technologies (i.e., inventing new devices or tools to support research or production), proposing unique solutions to existing problems (i.e., providing novel and effective solutions to known problems), combining existing methods in a novel way (i.e., combining existing methods or technologies in an innovative way to produce new effects), etc.

[0022] Through the predetermined quantification dimension, it is ensured that there are clear evaluation criteria for the scientific problems or research results to be evaluated in each key area, making the entire evaluation process more objective and systematic. Through the quantification in the four dimensions of theory, application, frontier and innovation, the various values of scientific problems can be comprehensively evaluated, and the value of scientific problems can be comprehensively understood from different perspectives, which helps decision-makers or researchers accurately judge the true value of technologies or research.

[0023] Step 2: Introduce the predetermined triple strategy to construct the target instruction fine-tuning dataset for the target quantification dimension.

[0024] Specifically, the predetermined triple strategy is a structured data organization method, which includes three core parts: the instruction unit, the input unit and the output unit, generating a standardized data format for the task. The instruction unit contains a clear task definition and supplements the definition descriptions of theoretical value, application value, frontier value and innovation value. The input unit contains sentence samples from a large number of scientific and technological literatures. The definitions of different types of value sentences provide clear task boundaries to assist the model in understanding the characteristics of different types of sentences. The output unit adopts the annotation method of "sentence type: sentence content", realizing fine-grained speech step mapping and providing the model with language features and context information. Obtain the target definition descriptions of the theoretical value dimension, application value dimension, frontier value dimension and innovation value dimension in the target quantification dimension, and combine them with the task boundary definition to generate clear task instructions for each dimension, jointly constituting the instruction unit.

[0025] Collect scientific problem samples related to the target quantification dimension from scientific and technological literatures, which can be from scientific research articles, technical reports, etc. in different fields, and store them in the input unit. Specify the corresponding quantification evaluation results and explanations for each input sample, the scoring of the target dimension (such as high, medium, low), and attach explanations. Combine the instruction unit, the input unit and the output unit into a standardized triple format. Each triple will represent a complete training sample, providing structured data for the fine-tuning of the model.

[0026] Through the above, a target instruction fine-tuning dataset containing multiple triples is constructed for identifying different types of scientific value sentences. With clear task definitions and detailed value type descriptions (theoretical value, application value, frontier value, innovation value), a clear task boundary is provided for the model. The structure of the target instruction fine-tuning dataset adopts the triple form, including task instructions, sentence input samples, and fine-grained output annotations, aiming to help the model accurately understand and identify different types of scientific value sentences in scientific literature. Through the triple strategy, the data format is standardized, enabling the model to process inputs and outputs consistently, reducing manual intervention and data bias, and comprehensively evaluating scientific issues from four dimensions (theory, application, frontier, innovation) to ensure that the model can more comprehensively understand and evaluate the value of research.

[0027] Step 3: Quantize the parameters of the target instruction fine-tuning dataset based on the QLoRA large language model fine-tuning principle to obtain a target recognition large model.

[0028] Specifically, to improve the quality of knowledge extraction, the QLoRA fine-tuning method is used to fine-tune the large model. QLoRA is an efficient large language model fine-tuning method that combines two techniques: quantization and low-rank adaptation. Its main purpose is to maintain the performance of the large language model while reducing memory occupancy. By quantizing the model parameters into a low-precision format (e.g., 4-bit) and introducing low-rank matrices to adapt to new tasks, QLoRA effectively solves the application problem of large models in resource-constrained environments. QLoRA first quantizes the weight matrix of the model into 4-bit to reduce memory and computational requirements; then, by introducing a low-rank adaptation matrix (usually with a smaller dimension) to correct the performance loss caused by quantization, the performance of the model is optimized.

[0029] Input the constructed target instruction fine-tuning dataset into a pre-trained large language model. The weights of the pre-trained model are usually floating-point numbers (such as 32-bit floating-point numbers). When performing fine-tuning, they are quantized into 4-bit format through the QLoRA method, significantly reducing memory usage while retaining the effectiveness of the model on new tasks. Parameter quantization is to convert high-precision floating-point numbers (e.g., 32-bit floating-point numbers) into low-precision formats (e.g., 4-bit integers) to reduce computational complexity and memory requirements. 4-bit quantization means compressing each parameter value into 4-bit data, usually by reducing the precision of the numbers.

[0030] The QLoRA method converts the parameters of the model (such as the weight matrix W of the i-th layer i ) from the original high-precision floating-point number format (such as 32-bit floating-point numbers) into 4-bit integers, and introduces a low-rank adaptation matrix (the first projection matrix A i and the second projection matrix Bi ) to correct the precision loss caused by quantization and enable the model to quickly adapt to new tasks. Through forward calculation correction and optimization, the QLoRA model can be effectively adapted in new tasks. By combining the original quantized model parameters with the low-rank adaptation matrix, the performance loss caused by quantization is corrected. In forward calculation, the model not only uses the quantized weight matrix for calculation but also uses the projection matrix for correction to ensure that the output of each layer accurately reflects the goals of the new task. After the quantization, correction, and optimization process, the model generates a large target recognition model that has been fine-tuned and adapted to the target task, which can show high accuracy on the task-related dataset and effectively execute tasks (such as the value evaluation of scientific questions). Through QLoRA, although the memory footprint of the model is greatly reduced, the performance is still guaranteed. Especially when dealing with large-scale datasets and large language models, the advantages of QLoRA are very obvious.

[0031] Step 4: Obtain the scientific question text, perform semantic feature analysis on the scientific question text through a gating network to obtain the text semantics, and obtain the target weight coefficient of the scientific question text on the target quantization dimension according to the text semantics.

[0032] Specifically, obtain the scientific question text from scientific research literature, technical reports, academic articles, or others. The scientific question text refers to the text containing the description of scientific questions, such as abstracts, sentences, or paragraphs describing a certain research topic, question, or challenge, which is usually the basis for evaluating its theoretical, application, frontiers, and innovation values. The scientific question text may be a paragraph or sentence in a scientific literature that describes a specific scientific research question. The gating network is a neural network structure used to control the information flow. It determines which information needs to be transmitted through the network and which information needs to be blocked through a gating mechanism. The gating network is often used to process complex input data and can apply different processing strategies in different parts of the network to improve the flexibility and effectiveness of the model.

[0033] The gating network receives the input scientific question text and performs semantic feature analysis on the text through a series of neural network layers, performing operations such as word segmentation, word embedding, and context modeling on the text to extract the important semantic information in the text. Semantic feature analysis refers to analyzing the grammar, semantics, and context information in the text to mine the important features related to the scientific question. The text semantics is usually a fixed-size vector, called the text semantic vector. After being processed by the gating network, a four-dimensional semantic feature vector (logit vector) is obtained, and each dimension represents the potential score of the text on a certain target quantization dimension. The four dimensions respectively represent the preliminary scores of theoretical value, application value, frontiers value, and innovation value.

[0034] Using the text semantic vector, the weight coefficient of the scientific question text in the target quantization dimension is predicted through a mapping layer or a regression model, that is, obtained by processing the semantic features of the text (such as using the Softmax function). The role of the Softmax function is to convert these original logit values into a probability distribution, making their sum equal to 1. The weight coefficient of each target dimension represents the relative importance of that dimension in the evaluation. The Softmax function is a commonly used normalization tool that can convert a logit vector into a probability distribution for handling multi-classification problems. By performing semantic feature analysis on the text through a gating network, the meaning of the scientific question text can be deeply understood, and accurate evaluation can be carried out from four dimensions (theory, application, frontier, innovation). Through Softmax normalization, it can be ensured that the scores of each dimension are comparable, and the sum of the results in all dimensions is 1, thus making the model output more stable and unified. By using the gating network and Softmax normalization, the model can automatically extract the weight coefficients of the target quantization dimension from the input scientific question text, avoiding manual intervention and subjective judgment, and improving the objectivity and efficiency of the evaluation process.

[0035] Step Five: Analyze the scientific question text through the target recognition large model and obtain the target quantization result in combination with the target weight coefficient.

[0036] Specifically, analyze the input scientific question text through the fine-tuned target recognition large model, and match the target recognition function corresponding to the target quantization dimension in the target recognition large model, that is, the specific function used by the model to evaluate the input text (scientific question text) in each quantization dimension. The input scientific question text will be processed by the target recognition large model and generate a target evaluation result, which is the score of each quantization dimension, representing the performance of the text in that dimension. Weight the evaluation results of each dimension according to the previously obtained target weight coefficient to obtain the target quantization result, that is, the comprehensive value score of the scientific question text, which synthesizes the performance of the text in all dimensions and is adjusted according to the importance (i.e., weight) of each dimension. By using the target recognition large model to analyze the scientific question text and obtaining the target quantization result in combination with the target weight coefficient, the importance and value of the scientific question are evaluated, which helps to identify high-value scientific questions.

[0037] Further, as shown in the appendix Figure 2 The second step of this application includes:

[0038] The predetermined triple strategy includes an instruction unit, an input unit, and an output unit; perform discriminative feature analysis on the theoretical value dimension, the application value dimension, the frontier value dimension, and the innovation value dimension to determine the task boundary definition; obtain the target definition description of the target quantization dimension, and combine the task boundary definition to form the instruction unit; construct the scientific question sample set of the target quantization dimension, and store the scientific question sample set in the input unit; read the predetermined annotation scheme, and store the predetermined annotation scheme in the output unit; construct the instruction unit, the input unit, and the output unit according to the predetermined triple strategy to obtain the target instruction fine-tuning data set.

[0039] Specifically, the predetermined triple strategy is a data construction scheme, where each triple consists of three parts: an instruction unit, an input unit, and an output unit, providing a structured and standardized data format for the training of the model. The instruction unit contains the clear instructions or descriptions of the task, telling the model what operations should be performed, usually including the definition and requirements of the task; the input unit contains the actual input data or samples, which are the information that the model needs to process when performing the task; the output unit defines the output results that the model should produce after processing the input data, usually the expected predictions or labels.

[0040] Construct a target instruction fine-tuning data set containing four quantization dimensions (theoretical value, application value, frontier value, innovation value) according to the predetermined triple strategy. First, perform discriminative feature analysis on the four dimensions (theoretical, application, frontier, innovation value) to clarify the evaluation criteria and requirements of each dimension, and understand the characteristics of each dimension and its role in the task. The discriminative feature analysis is a detailed analysis of each quantization dimension to determine the unique features and evaluation criteria, so as to clarify the task boundary. The task boundary definition is to clarify the scope and objectives of the task, defining the input, output of the task, and the features or attributes that the model needs to focus on.

[0041] The target definition description is a detailed explanation of each dimension in the target quantization dimension, clarifying the specific meaning and evaluation criteria of each dimension. Based on the task boundary definition, further obtain and describe the definition of each quantization dimension to ensure that the model can accurately understand the task requirements, that is, the evaluation criteria of each value dimension explained above. In the instruction unit, it contains a clear task description, such as "given a scientific abstract, evaluate its innovation value, identify original discoveries, method innovations, or novel applications and supplement the definition descriptions of the theoretical value, application value, frontier value, and innovation value".

[0042] Next, collect text samples of scientific questions related to these target quantification dimensions to form a scientific question sample set. The scientific question sample set contains sentences from various types of scientific and technological literature, all covering different characteristics of the above four dimensions. Each sample needs to be annotated to indicate the corresponding quantification dimension (theory, application, frontier, innovation) and its evaluation result. Store the scientific question sample set in the input unit, and diverse training data can be received during fine-tuning. The definitions of different types of value sentences provide clear task boundaries to assist the model in understanding the characteristics of different types of sentences. In the input field, the data set contains a large number of sentence samples from scientific and technological literature, such as descriptions of a certain technical solution or research, covering various sentence pattern characteristics in scientific and technological literature.

[0043] Read a predetermined annotation scheme, that is, pre-defined annotation rules or standards for annotating data. The annotation form of "sentence type: sentence content" is adopted to achieve fine-grained speech act mapping, providing language features and context information for the model. For example: Theoretical value: medium, reasons: 1. Verified existing theoretical predictions; 2. Contributed to the theory of quantum computing; 3. Provided mathematical verification.

[0044] According to the predetermined triple strategy, combine the instruction unit, input unit, and output unit to obtain the target instruction fine-tuning data set. For example, a target instruction fine-tuning data set is constructed and a scientific question is evaluated. Suppose the scientific question text used is: "This study proposes a new artificial intelligence algorithm that can effectively improve the accuracy of image recognition and can process large-scale data in real time." Perform data annotation according to the predetermined triple strategy: Instruction unit: The task description is "evaluate the theoretical value, application value, frontier value, and innovation value of this study." Input unit: Scientific question text: "This study proposes a new artificial intelligence algorithm that can effectively improve the accuracy of image recognition and can process large-scale data in real time." Output unit: Theoretical value: medium (provides a new method but does not involve a new theoretical framework); Application value: high (can solve practical problems in the field of image recognition and has practical application potential); Frontier value: medium (although a new algorithm is applied, it does not involve revolutionary technological breakthroughs); Innovation value: high (proposes a new algorithm and is innovative).

[0045] By using the triple strategy, all data is standardized into triples with a clear structure, making data processing and model training more consistent and reproducible, providing multi-dimensional value evaluations for each scientific question, not only focusing on a single dimension such as citation frequency or h-index, but comprehensively evaluating its theoretical, application, frontier, and innovation values, combining the instruction unit, input unit, and output unit, accurately understanding the characteristics of each dimension, and making reasonable evaluation results.

[0046] Furthermore, step three of this application includes:

[0047] Obtain a pre-trained model, where the pre-trained model includes the original weight matrix \(W\) of the \(i\)-th layer i ; obtain a predetermined projection matrix, where the predetermined projection matrix includes a first projection matrix \(A\ i and a second projection matrix \(B\ i , and the dimension of the first projection matrix \(A\ i is \((d, r)\), and the dimension of the second projection matrix \(B\ i is \((r, d)\), where \(d\) is the hidden layer dimension and \(r\) is the projection dimension, and \(r\) is much smaller than \(d\); combine the first projection matrix \(A\ i and the second projection matrix \(B\ i to perform forward calculation correction and optimization on the original weight matrix \(W\) quantized into 4-bit format i to obtain the target recognition large model.

[0048] Specifically, obtain a pre-trained model, which has already been trained on a large-scale dataset and has effective parameters and weights. A pre-trained model refers to a model that has been trained on a large-scale dataset and has learned the latent patterns and features in the data. During the fine-tuning process, the original weight matrix of the pre-trained model will be used as a basis for modification to adapt to a specific task. Extract the original weight matrix \(W\) of the \(i\)-th layer from the pre-trained model i , that is, the object to be fine-tuned.

[0049] Obtain a predetermined projection matrix, including a first projection matrix \(A\ i and a second projection matrix \(B\ i , the dimension of the first projection matrix \(A\ i is \((d, r)\), which is used to project the high-dimensional weight matrix into a low-dimensional space; the second projection matrix \(B\ i has a dimension of \((r, d)\) and is used to re-project the representation in the low-dimensional space back to the original high-dimensional space; where \(d\) is the hidden layer dimension of the model, representing the size of the feature space of this layer of neural network; \(r\) is the projection dimension, usually much smaller than \(d\), for example, \(r\) may only be a few hundred or a few thousand, while \(d\) can be several thousand or several ten thousand.

[0050] The role of the projection matrix is to map the large-scale weight matrix to a low-dimensional space through the first projection matrix and then map it back to the original space through the second projection matrix from the low-dimensional space, which can not only reduce the calculation and storage costs but also maintain sufficient expressive power during the training process. To further reduce the memory occupancy, the original weight matrix \(W\ iQuantized into a 4-bit format, that is, each parameter value is compressed from a 32-bit or 64-bit floating-point number to a 4-bit representation, significantly reducing the memory usage. The quantized weight matrix is stored in a more compact manner and improves the computational and storage efficiency without sacrificing the model accuracy. Combine the quantized original weight matrix with the first projection matrix A i and the second projection matrix B i Combine, use the first projection matrix A i to project the quantized weight matrix into a low-dimensional space, which can be achieved through matrix multiplication. Use the second projection matrix B i to restore the representation in the low-dimensional space to the original dimension, which is also completed through matrix multiplication.

[0051] Perform forward calculation, that is, use the corrected weight matrix for the forward propagation of the neural network. The quantized original weight matrix W i has been converted from a high-precision floating-point number (such as 32-bit) to a 4-bit format, greatly reducing the storage space of the model, but also introducing quantization errors. To correct the accuracy loss due to quantization, QLoRA introduces the low-rank adaptation technique. By introducing the first projection matrix A i and the second projection matrix B i , QLoRA can effectively correct the quantized model weights, reduce errors, and improve performance. The dimensions (d, r) and (r, d) of the projection matrix represent its adaptability and compressibility. r is much smaller than d, so the projection matrix only introduces a small amount of additional computational overhead. When performing forward calculation, the quantized weight matrix W i is combined with the projection matrices A i and B i for calculation correction. The output of the quantized model can recover some of the errors caused by quantization, ensuring that the model still performs well in new tasks. Combining the correction results of the quantized weight matrix and the projection matrix, after completing the forward calculation, the output of the model will be used for object recognition tasks (such as text classification, question answering, etc.). Through the optimization of the training data, QLoRA can generate a large object recognition model that performs well on the target task, that is, a large model that can effectively perform recognition tasks after quantization and low-rank adaptation optimization.

[0052] Furthermore, this application also includes the following steps:

[0053] Read the forward calculation correction function, where the expression of the forward calculation correction function is as follows: h i = f i (x i ); where, h i refers to the original forward calculation function of the i-th layer, and x iRefers to the scientific problem text i, Refers to the original weight matrix W quantized into a 4-bit format i Performs weight dequantization to floating point, Δ i = A i B i f i (x i ) refers to the low-rank correction term introduced by QLoRA.

[0054] Specifically, forward computation refers to the process of obtaining an output after the input data passes through the operations of neurons in each layer. The forward computation correction function refers to a function that corrects and optimizes the results in this computation process. In the QLoRA method, the correction function is used to correct the quantized model weights to improve the accuracy and performance of the model. The expression of the forward computation correction function is as follows: h i = f i (x i ); The error caused by quantization is corrected through low-rank adaptation, thereby improving the accuracy of the fine-tuned model. Among them, h i refers to the original forward computation function of the i-th layer, which is calculated through the forward computation function f i (x i ), that is, the result obtained by calculating the i-th layer through the input scientific problem text x i ; x i refers to the scientific problem text i, that is, the input data, which is a certain text (such as a sentence in a scientific paper) here and is processed through a neural network to obtain the corresponding output.

[0055] Refers to the original weight matrix W quantized into a 4-bit format i Performs weight dequantization to floating point. The quantization operation compresses the values of the original weight matrix into a 4-bit representation (low precision), and dequantization restores these 4-bit quantized values to the original floating point values. Since the QLoRA method quantizes the weight matrix to 4-bit, it is necessary to restore the quantized weight matrix to the floating point format in each calculation; Δ i = A i B i f i (x i ) refers to the low-rank correction term introduced by QLoRA, which is jointly calculated by two projection matrices A i and B i and the forward computation function f i (x i ). Adding the low-rank correction term Δ i to the original forward computation result h i, obtain a new output h i ′, representing the output of the i-th layer after correction, which includes the effects of the dequantized weights and the low-rank correction terms. After the forward calculation correction, the new model obtained will better adapt to a specific task and be able to perform a more accurate analysis of the input data. During the fine-tuning process, the forward calculation correction function helps the model maintain a high calculation accuracy while being quantized.

[0056] Furthermore, the present application further includes the following steps:

[0057] Obtain a predetermined loss function; perform forward calculation correction optimization with the minimum loss value of the predetermined loss function as the optimization goal to obtain the target recognition large model; wherein, the expression of the predetermined loss function is as follows: where, θ 4bit refers to the quantized original model parameters, refers to all the projection matrices introduced by QLoRA, D is the training data set for the new task, l is the task-related loss function, and during the optimization process, only is updated while keeping θ 4bit unchanged.

[0058] Specifically, the loss function is used to measure the gap between the prediction result of the model and the true label. The predetermined loss function refers to the loss function preset during the model training process, which is used to evaluate the performance of the model and guide the optimization process. The loss function in the QLoRA optimization process is used to minimize the prediction error of the quantized model. Predetermined loss function: where, θ 4bit refers to the quantized original model parameters, refers to all the projection matrices introduced by QLoRA, D is the training data set for the new task, l is the task-related loss function, which is used to calculate the error between the model output and the true label; during the optimization process, only is updated while keeping θ 4bit unchanged. is the target loss function, which measures the performance of the quantized model and the corrected projection matrix on the new task. (x, y) is a pair of samples in the training data set D, x is the input data (such as scientific question text), and y is the target output (such as text classification label or regression value). represents the forward calculation output of the quantized original model and all the projection matrices on the input x.

[0059] During the optimization process of QLoRA, the goal is to minimize the loss function L, that is, to optimize the performance of the model on the new task. The core of the QLoRA method is to keep the quantized weight matrix θ 4bit unchanged and only update Make the corrected model able to produce the minimum loss on the training dataset of the new task. For example, assume that QLoRA is used to fine-tune a quantized BERT model. The original weight matrix θ of the i-th layer of the model 4bit has already been quantized to 4-bit. The training dataset D contains 1000 sample pairs (x, y), where x is the text paragraph of a scientific paper and y is the corresponding label (such as a classification label). The cross-entropy loss function is used as the task-related loss function l. During the optimization process, the quantized model parameters θ 4bit remain unchanged. Assume that the dimensions of the projection matrices are 1024×128 and 128×1024 respectively. The projection matrices are gradually updated by the gradient descent method to minimize the loss function and optimize the projection matrices. By optimizing all the introduced projection matrices, the QLoRA method can effectively correct the errors caused by quantization, enabling the quantized model to still maintain a high accuracy on the new task.

[0060] Furthermore, step four of the present application includes:

[0061] Perform a feed-forward mapping on the scientific question text through the gating network to obtain a four-dimensional logit vector; normalize the four-dimensional logit vector using Softmax to obtain the target weight coefficients.

[0062] Specifically, the gating network processes and maps the scientific question text. Through the feed-forward mapping process, the gating network extracts useful semantic information from the input text and converts it into a four-dimensional logit vector. In natural language processing tasks, the gating network is used to select and transmit useful information through a weighting mechanism, helping the model focus on the most critical parts of the input data. The structure of the gating network usually includes multiple gates that control different information flows, such as input gates, forget gates, output gates, etc. Each gate determines the transmission intensity of information through learned weights. Logit refers to the raw prediction value output by the neural network, and logit is usually the raw score of each category predicted by the model. The four-dimensional logit vector means that the network outputs a four-dimensional vector, where each dimension corresponds to the score of the scientific question text in a certain dimension (such as theoretical value, application value, frontier value, and innovation value). Feed-forward mapping refers to the layer-by-layer calculation process from the input data (such as scientific question text) to the output layer. In a feed-forward network, information is transmitted through the network layers in a one-way manner, and finally the corresponding output is generated. Each layer's input vector will go through weighting, non-linear activation, and further processing, and finally form the output.

[0063] After the input scientific problem text is processed by the feed-forward mapping and weighted by the gating mechanism, the gating network can automatically adjust its information processing method according to different task requirements. The four-dimensional logit vector generated by the feed-forward mapping reflects the preliminary evaluation of the scientific problem text in four quantification dimensions. These logit values are not the final evaluation results but the preliminary prediction scores of the text. Next, these logit values can be further processed (such as Softmax normalization) to obtain the target weight coefficients for a more accurate quantitative evaluation.

[0064] The Softmax function normalizes the above four-dimensional logit vector and converts it into a probability distribution. Through Softmax normalization, the logit value of each dimension will be converted into a value between 0 and 1, and the sum of the target weight coefficients of all four dimensions will be 1. Usually, the process of normalization calculation includes: for each element in the logit vector, first calculate the exponent, then obtain the sum of the exponent values, calculate the probability value of each logit, and obtain the normalized target weight coefficient. The normalized target weight coefficient can convert the score of each dimension into a probability distribution, facilitating further analysis and decision-making, making the evaluation results of the text more standardized and unified, and avoiding the problem that the score values of different dimensions are not comparable.

[0065] For example, if the four-dimensional logit vector is [2.3, 1.5, 0.7, 1.2], the result after Softmax processing may be [0.40, 0.30, 0.15, 0.15], that is, the target weight coefficients are 0.40, 0.30, 0.15, and 0.15 respectively, indicating that the theoretical value is the most important, followed by the application value, while the frontier value and innovation value are relatively less important in this text. After Softmax normalization, the obtained target weight coefficients represent the relative importance of the scientific problem text in the four value dimensions. Through the feed-forward mapping of the gating network, multi-dimensional information is extracted from the scientific problem text and converted into a logit vector. Through Softmax normalization, the original scores of the scientific problem in the four dimensions can be converted into target weight coefficients in probability form, thereby quantifying the relative importance of each dimension and more clearly identifying the contributions of the scientific problem in each value dimension in the evaluation task.

[0066] Furthermore, step five of this application includes:

[0067] Matching the target recognition function corresponding to the target quantification dimension in the target recognition large model; analyzing the scientific problem text through the target recognition function to obtain the target evaluation result; calculating the target evaluation result and the target weight coefficient to obtain the target quantification result.

[0068] Specifically, the large target recognition model (the model obtained by fine-tuning with QLoRA) matches corresponding target recognition functions according to target quantization dimensions (such as theoretical value, application value, frontier value, and innovation value). The target recognition function is a mathematical function that, based on the trained model parameters, is used to calculate the score of a scientific problem text in a specific dimension. For example, in the dimension of theoretical value, the target recognition function may evaluate based on features such as theoretical innovation, hypothesis verification, and theoretical extension in the text. The target recognition function is a text analysis function for different quantization dimensions (such as theoretical value, application value, etc.). The target recognition function for each dimension is usually obtained by a deep learning model through a large amount of training, which can extract specific features in the text and give a score.

[0069] Using the matched target recognition function, the input scientific problem text is analyzed to obtain the target evaluation result. Each target recognition function processes the key information in the text and outputs a corresponding score. By processing the text with the target recognition function, one or more evaluation scores are generated, representing the performance of the text in each quantization dimension. The target evaluation result is a numerical score for each dimension, indicating the value of the text in that dimension. For example, suppose the evaluation result given by the target recognition function for application value is 3.8 points, and the evaluation result for innovation value is 4.5 points.

[0070] The target weight coefficient is the coefficient obtained by normalizing through the Softmax function, indicating the importance of each evaluation dimension in target quantization. The target evaluation result and the target weight coefficient are weighted and summed. That is, the evaluation score for each dimension is multiplied by the previously calculated target weight coefficient, and the product results for all dimensions are weighted and summed to obtain the target quantization result, reflecting the comprehensive value of the scientific problem text in all target dimensions. Through multi-dimensional analysis of the text by the target recognition function and in combination with the weight coefficient, the scientific value of the text can be comprehensively evaluated, not limited to a single dimension, avoiding manual subjectivity. Through weighted calculation, the information in different dimensions is effectively integrated together to obtain a comprehensive target quantization result, making the evaluation more comprehensive and avoiding over-dependence on or neglect of a certain dimension.

[0071] In summary, the scientific problem value quantization method with an embedded multi-dimensional hybrid expert mechanism provided in this application has the following technical effects:

[0072] By reading a predetermined quantization dimension, the predetermined quantization dimension includes a target quantization dimension; introducing a predetermined triple strategy to construct a target instruction fine-tuning dataset for the target quantization dimension; performing parameter quantization on the target instruction fine-tuning dataset based on the QLoRA large language model fine-tuning principle to obtain a target recognition large model; obtaining a scientific question text, performing semantic feature analysis on the scientific question text through a gating network to obtain text semantics, and obtaining a target weight coefficient of the scientific question text on the target quantization dimension according to the text semantics; analyzing the scientific question text through the target recognition large model and combining the target weight coefficient to obtain a target quantization result. That is, through a clear predetermined quantization dimension, including multiple aspects such as theoretical value, application value, frontier value, and innovation value, using the triple strategy to construct a dedicated dataset, using the QLoRA large language model fine-tuning principle for parameter quantization, introducing a gating network to automatically analyze the key information of the text, determining the weight coefficients of the scientific question text on each target quantization dimension, and obtaining a quantified evaluation result, which improves the efficiency of the value quantization evaluation of scientific questions.

[0073] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0074] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application also intends to include these changes and modifications.

Claims

1. A scientific problem value quantification method embedded in a multi-dimensional hybrid expert mechanism is characterized by: include: Reading predetermined quantization dimensions, where the predetermined quantization dimensions include target quantization dimensions; Introducing a predetermined triplet strategy to construct a target instruction fine-tuning dataset of the target quantization dimension; Based on the QLoRA large language model fine-tuning principle, the target instruction fine-tuning dataset is parameterized to obtain a target recognition large model; Obtaining a scientific question text, performing semantic feature analysis on the scientific question text through a gating network to obtain text semantics, and obtaining a target weight coefficient of the scientific question text on the target quantitative dimension based on the text semantics; The scientific question text is analyzed by the target recognition model, and the target quantification result is obtained by combining the target weight coefficient.

2. The method for quantifying the value of scientific problems embedded in a multi-dimensional hybrid expert mechanism according to claim 1 is characterized in that: The predetermined quantitative dimensions include at least theoretical value dimension, application value dimension, frontier value dimension and innovation value dimension.

3. The method for quantifying the value of scientific problems embedded in a multi-dimensional hybrid expert mechanism according to claim 2 is characterized in that: The predetermined triplet strategy includes an instruction unit, an input unit, and an output unit. Introducing the predetermined triplet strategy to construct a target instruction fine-tuning dataset of the target quantization dimension includes: Conduct distinguishing characteristic analysis on the theoretical value dimension, the applied value dimension, the frontier value dimension, and the innovative value dimension to determine the definition of task boundaries; Obtaining a target definition description of the target quantization dimension and combining it with the task boundary definition to form the instruction unit; assembling a sample set of scientific questions of the target quantitative dimension, and storing the sample set of scientific questions in the input unit; Reading a predetermined marking scheme, and storing the predetermined marking scheme in the output unit; The instruction unit, the input unit, and the output unit are constructed according to the predetermined triple strategy to obtain the target instruction fine-tuning data set.

4. The method for quantifying the value of scientific problems embedded in a multi-dimensional hybrid expert mechanism according to claim 1 is characterized in that: Based on the QLoRA large language model fine-tuning principle, the target instruction fine-tuning dataset is parameterized to obtain a large target recognition model, including: Get a pre-trained model, which includes the original weight matrix W of the i-th layer i ; Obtain a predetermined projection matrix, wherein the predetermined projection matrix includes a first projection matrix A i and the second projection matrix B i , and the first projection matrix A i The dimension is (d, r), the second projection matrix B i The dimension is (r, d), where d is the hidden layer dimension and r is the projection dimension, and r is much smaller than d; Combined with the first projection matrix A i and the second projection matrix B i The original weight matrix W quantized to 4-bit format i Perform forward calculation correction and optimization to obtain the target recognition large model.

5. The method for quantifying the value of scientific problems embedded in a multi-dimensional hybrid expert mechanism according to claim 4 is characterized in that: Read the forward calculation correction function, where the expression of the forward calculation correction function is as follows: h i =f i (x i ); Among them, h i Refers to the original forward calculation function of the i-th layer, x i refers to the scientific question text i, Refers to the original weight matrix W that is quantized into 4-bit format i Dequantize the weights into floating point numbers, Δ i =A i B i f i (x i ) refers to the low-rank correction term introduced by QLoRA.

6. The method for quantifying the value of scientific problems embedded in a multi-dimensional hybrid expert mechanism according to claim 5 is characterized in that: Based on the QLoRA large language model fine-tuning principle, the target instruction fine-tuning dataset is parameterized to obtain a large target recognition model, including: Get the predetermined loss function; Perform forward calculation correction optimization with the minimum loss value of the predetermined loss function as the optimization goal to obtain the target recognition large model; The expression of the predetermined loss function is as follows: Among them, θ 4bit refers to the original model parameters after quantization, Refers to all projection matrices introduced by QLoRA, D is the training dataset of the new task, l is the task-related loss function, where, during the optimization process, only update While keeping θ 4bit constant.

7. The method for quantifying the value of scientific problems embedded in a multi-dimensional hybrid expert mechanism according to claim 1 is characterized in that: Obtaining a scientific question text, performing semantic feature analysis on the scientific question text through a gating network to obtain text semantics, and obtaining a target weight coefficient of the scientific question text on the target quantitative dimension based on the text semantics, including: Perform feed-forward mapping on the scientific question text through the gating network to obtain a four-dimensional logit vector; The four-dimensional logit vector is normalized using Softmax to obtain the target weight coefficient.

8. The method for quantifying the value of scientific problems embedded in a multi-dimensional hybrid expert mechanism according to claim 1 is characterized in that: The scientific question text is analyzed by the target recognition model, and the target quantification result is obtained by combining the target weight coefficient, including: Matching the target recognition function corresponding to the target quantization dimension in the target recognition large model; Analyzing the scientific question text by using the target identification function to obtain a target evaluation result; The target evaluation result and the target weight coefficient are calculated to obtain the target quantification result.

Images