An Adaptive Knowledge Extraction Method and System for Scientific and Technical Literature

Abstract

The present invention discloses an adaptive knowledge extraction method and system for scientific and technological literature, which relates to the field of knowledge extraction technology, including: receiving a task request and target document content input by a user; extracting key information from the task request to obtain a semantic parsing result; mapping the semantic parsing result to a task type to obtain multiple knowledge extraction tasks; matching model weights according to the multiple knowledge extraction tasks to obtain multiple task model weights; dynamically loading the task model weights into a general basic model to complete the initialization of a task-specific model; extracting knowledge from the target document content, outputting multiple original extraction results, and formatting them to obtain multiple formatted results, which are then output and displayed to the user. The present invention solves the technical problems of the prior art in that the adaptability is poor in multi-task scenarios and the accuracy of the knowledge extraction results is low, thereby achieving the technical effect of improving the adaptability to multi-task scenarios and improving the accuracy of the knowledge extraction results.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge extraction, and in particular to an adaptive knowledge extraction method and system for scientific and technological literature. Background Art

[0002] Knowledge extraction from scientific literature is an important means of processing and extracting key information from documents. It is widely used in fields such as scientific research literature analysis, academic search engines, and intelligent question-answering systems. However, due to the complex content, diverse structure, and wide range of fields covered by scientific literature, traditional knowledge extraction methods often suffer from poor adaptability and low extraction accuracy in multi-task scenarios. These issues not only affect the efficiency and accuracy of knowledge extraction, but also increase the difficulty of model training and adaptation.

[0003] Currently, existing knowledge extraction methods primarily rely on models and algorithms designed for specific tasks, such as rule-based extraction methods, machine learning models, and deep learning methods. These methods have achieved some success in specific tasks, but when faced with multi-task, cross-domain scientific literature processing, they often lack good generalization and adaptability between tasks. This results in the model being unable to share knowledge and experience across multiple tasks, reducing the overall effectiveness of the system. Summary of the Invention

[0004] The present application provides an adaptive knowledge extraction method and system for scientific and technological literature, which is used to solve the technical problems of poor adaptability of existing technologies in multi-task scenarios and low accuracy of knowledge extraction results.

[0005] In view of the above problems, the present application provides an adaptive knowledge extraction method and system for scientific and technological literature.

[0006] The first aspect of the present application provides an adaptive knowledge extraction method for scientific and technological literature, the method comprising:

[0007] Receive a task request and target document content input by a user; perform key information extraction by semantically parsing the task request to obtain a semantic parsing result; use predefined mapping rules to map the semantic parsing result to a task type to obtain a plurality of knowledge extraction tasks; perform model weight matching according to the plurality of knowledge extraction tasks to obtain a plurality of task model weights; pre-call a general basic model, and complete the initialization of a plurality of task-specific models by dynamically loading the plurality of task model weights into the general basic model; use the plurality of task-specific models to perform knowledge extraction on the target document content, and output a plurality of original extraction results; after formatting the plurality of original extraction results to obtain a plurality of formatted results, output and display the plurality of formatted extraction results to the user.

[0008] The second aspect of the present application provides an adaptive knowledge extraction system for scientific and technological literature, the system comprising:

[0009] An information receiving unit receives a task request and target document content input by a user; a semantic parsing unit extracts key information by performing semantic parsing on the task request to obtain a semantic parsing result; a task type mapping unit maps the semantic parsing result to a task type using a predefined mapping rule to obtain a plurality of knowledge extraction tasks; a weight matching unit performs model weight matching based on the plurality of knowledge extraction tasks to obtain a plurality of task model weights; a model initialization unit pre-calls a general basic model and completes the initialization of a plurality of task-specific models by dynamically loading the plurality of task model weights on the general basic model; a knowledge extraction unit extracts knowledge from the target document content using the plurality of task-specific models and outputs a plurality of original extraction results; a formatting processing unit formats the plurality of original extraction results to obtain a plurality of formatted results and then outputs and displays the plurality of formatted extraction results to the user.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] The present application receives a task request and target document content input by a user; extracts key information by semantically parsing the task request and obtains a semantic parsing result; uses predefined mapping rules to map the task type of the semantic parsing result and obtains multiple knowledge extraction tasks; performs model weight matching according to multiple knowledge extraction tasks and obtains multiple task model weights; pre-calls a general basic model and initializes multiple task-specific models by dynamically loading multiple task model weights on the general basic model; uses multiple task-specific models to extract knowledge from the target document content and outputs multiple original extraction results; formats the multiple original extraction results and obtains multiple formatted results, then outputs and displays multiple formatted extraction results to the user. The present invention solves the technical problems of poor adaptability and low accuracy of knowledge extraction results in the existing technology under multi-task scenarios. By combining methods such as task parsing, mapping rule matching, and dynamic loading of model weights, the general basic model and the task-specific model are combined to realize knowledge extraction and formatted display of the target document content, accurately extract key information, and achieve the technical effect of improving adaptability to multi-task scenarios and improving the accuracy of knowledge extraction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A flowchart of an adaptive knowledge extraction method for scientific literature provided in an embodiment of the present application;

[0014] Figure 2 A schematic diagram of the structure of an adaptive knowledge extraction system for scientific and technological literature provided in an embodiment of the present application.

[0015] Explanation of the accompanying symbols: information receiving unit 11, semantic parsing unit 12, task type mapping unit 13, weight matching unit 14, model initialization unit 15, knowledge extraction unit 16, formatting processing unit 17. DETAILED DESCRIPTION

[0016] This application provides an adaptive knowledge extraction method and system for scientific and technological literature, aiming to solve the technical problems of poor adaptability and low accuracy of knowledge extraction results in existing technologies in multi-task scenarios. Through task analysis, mapping rule matching, dynamic loading of model weights and other methods, combined with general basic models and task-specific models, the application realizes knowledge extraction and formatted display of the target document content, accurately extracts key information, and achieves the technical effect of improving the adaptability to multi-task scenarios and improving the accuracy of knowledge extraction results.

[0017] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0018] It should be noted that any variations of the terms "include" and "have" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products or devices.

[0019] Example 1, as Figure 1 As shown, the present application provides an adaptive knowledge extraction method for scientific and technological literature, the method comprising:

[0020] Step S100: receiving a task request and target document content input by a user.

[0021] In the embodiment of the present application, during the initial stage of knowledge extraction, a task request and target document content are received from the user. The user submits the task request in natural language, such as "extract research questions related to neural network optimization from the document," clearly describing the target information and content range to be extracted. At the same time, the user provides the target document content as the source of scientific and technological literature data to be processed. This content can be textual information such as the full text of the document, paragraphs, abstracts, or titles.

[0022] Step S200: extracting key information by semantically parsing the task request to obtain a semantic parsing result.

[0023] In an embodiment of the present application, by performing semantic parsing on the task request, the Agent module is used to perform semantic analysis on the natural language task request input by the user, and key information in the request is gradually extracted. This key information includes the specific content that the user wants to obtain, that is, the query target (such as "research method"), the clear query scope (such as "in this article"), and the specific constraints defined by the user (such as "related to deep learning"). After extracting this information, it is organized in a structured form to generate a semantic parsing result.

[0024] Furthermore, in the method provided in the embodiment of the application, key information is extracted by semantically parsing the task request to obtain a semantic parsing result, and the method further includes:

[0025] The task request is semantically analyzed by using an Agent module to extract key request information, wherein the key request information includes a query target, a query scope, and specific constraints; the key request information is organized into structured data, and the semantic parsing result is output.

[0026] In an embodiment of the present application, by using the Agent module, the task request submitted by the user is first received and a semantic analysis is performed on it. During the analysis process, the Agent module extracts key information from the task request through semantic understanding. This information includes the query target, query scope, and specific constraints. The query target is the main type of information that the user wants to extract from the literature, such as "research methods" or "research questions". The query scope limits the processing boundaries of the task, such as the "full text" or "abstract" section. Specific constraints are additional restrictions attached to the task, such as "related to deep learning" or "within a specific time period". After extracting these key information, the Agent module organizes them into structured data to ensure the standardized expression of each information point. Finally, the structured data is output as a semantic parsing result.

[0027] Step S300: Using predefined mapping rules to perform task type mapping of the semantic parsing result to obtain multiple knowledge extraction tasks.

[0028] In an embodiment of the present application, predefined mapping rules are first constructed based on the mapping logic between K basic knowledge types and task types. Subsequently, knowledge types are matched against the mapping rules by the query target in the semantic parsing results to generate a knowledge type list. The knowledge type list is then filtered based on the query range in the semantic parsing results, and a range filter list is output. Finally, the range filter list is further filtered based on specific constraints to generate a target list. Through this series of logical mapping and filtering, multiple knowledge extraction tasks are ultimately generated.

[0029] Furthermore, in the method provided in the embodiment of the application, predefined mapping rules are used to map the semantic parsing results to task types to obtain multiple knowledge extraction tasks, and the method further includes:

[0030] According to the mapping logic of K basic knowledge types and task types, the predefined mapping rules are constructed; based on the query target in the semantic parsing result, the knowledge types are matched with the predefined mapping rules, and a knowledge type list is output; the query scope in the semantic parsing result is used to perform range screening of the knowledge type list, and a range screening list is output; the specific constraint conditions in the semantic parsing result are used to perform constraint condition screening of the range screening list, and a target list is output; and the multiple knowledge extraction tasks are generated according to the target list.

[0031] In the embodiment of the present application, predefined mapping rules are first constructed based on the logical relationship between K basic knowledge types (such as research questions, research objectives, research background, research methods, research results, and keywords) and task types. These mapping rules clearly define the association between each knowledge type and the corresponding task. For example, the basic knowledge type "research methods" can correspond to the "method identification" task, and the "research question" can correspond to the "question sentence extraction" task.

[0032] Next, preliminary knowledge type screening is completed through query target matching. Using semantic rule-based matching technology, the query target in the semantic parsing results is compared against the mapping rules one by one. For example, if the query target is "research methods," the matching "method" knowledge type is extracted from the rule base to generate a preliminary list of knowledge types. This step ensures that the core requirements in the semantic parsing results are accurately identified.

[0033] The list of knowledge types is then further filtered based on the query scope in the semantic parsing results. The query scope reflects the operational boundaries of the user task, such as "in this article" or "in the abstract." Through rule filtering, the applicable scopes of knowledge types are compared, and knowledge types that meet the scope requirements are retained, thus generating a scope-filtered list. For example, if the scope is limited to "in the abstract," only knowledge types applicable to the abstract are retained.

[0034] Next, the scope filter list is further filtered based on specific constraints in the semantic parsing results. Specific constraints refer to additional restrictions within the task requirements, such as "related to deep learning." In this step, rule-based filtering is used to match each task type in the filter list based on the conditional phrases in the semantic parsing results. Task types that do not meet the specified conditions are eliminated, ultimately generating a target list.

[0035] Finally, multiple knowledge extraction tasks are generated based on the target list. These tasks clearly define the target content, operation scope, and additional conditions, such as "extracting research methods related to deep learning from the full text" or "identifying research questions in the abstract."

[0036] Step S400: performing model weight matching according to the plurality of knowledge extraction tasks to obtain a plurality of task model weights.

[0037] In an embodiment of the present application, when matching model weights according to multiple knowledge extraction tasks, the corresponding number of training samples is first set for N types of knowledge extraction tasks, and N instruction fine-tuning data sets are constructed based on this as a constraint condition; then, based on these fine-tuning data sets, QLoRA fine-tuning training is performed on the general basic model to generate N task model weights respectively, and finally multiple task model weights corresponding to multiple knowledge extraction tasks are obtained.

[0038] Furthermore, in the method provided in the embodiment of the application, model weight matching is performed according to the multiple knowledge extraction tasks to obtain multiple task model weights, and the method further includes:

[0039] Set N numbers of training samples for N kinds of knowledge extraction tasks; construct N instruction fine-tuning data sets for the N kinds of knowledge extraction tasks with the N numbers of training samples as constraints; perform QLoRA fine-tuning training on the universal basic model based on the N instruction fine-tuning data sets to generate N task model weights for the N kinds of knowledge extraction tasks respectively; and dynamically call the multiple task model weights corresponding to the multiple knowledge extraction tasks from the N task model weights.

[0040] In this example, we first set the number of training samples for each of N knowledge extraction tasks (e.g., "extracting research methods" and "identifying research questions"). These sample numbers are determined based on the complexity and semantic characteristics of the tasks and serve as the basic constraints for constructing the fine-tuning dataset. For example, the "research methods" task might require 50,000 samples, while the "keyword extraction" task might require 30,000 samples.

[0041] Then, based on the above number of samples, N instruction fine-tuning datasets are constructed. Each dataset is in the form of a triple, containing "instructions" (clear task definition, such as "identify research methods"), "input" (sentences or paragraphs of literature content), and "output" (standardized results of task objectives). For example, for the "research methods" task, the input is "This paper proposes a new algorithm", and the output is "True / False", indicating whether it is a sentence related to research methods. In this way, the fine-tuning dataset covers the diverse expressions of the task and provides the model with clear guidance on the task objectives.

[0042] The general base model is then trained using QLoRA fine-tuning based on N instruction fine-tuning datasets. QLoRA technology quantizes model weights to 4-bit format to reduce video memory requirements and introduces a low-rank adaptation matrix to optimize model parameters, enabling efficient adaptation of the model to specific tasks. During training, model weights are independently adjusted for each task dataset to optimize performance on that task, ultimately generating N task model weights. Each weight contains task-specific fine-tuning parameters specifically designed to perform the corresponding knowledge extraction task.

[0043] Finally, multiple task model weights corresponding to multiple knowledge extraction tasks are dynamically called from N task model weights. For example, when processing the "extract research method" task, the "method identification task weight" is dynamically loaded from the weight library; and when executing the "keyword extraction" task, the "keyword extraction task weight" is dynamically loaded.

[0044] Furthermore, in the method provided in the embodiment of the application, QLoRA fine-tuning training is performed on the general basic model based on the N instruction fine-tuning data sets to generate N task model weights for the N knowledge extraction tasks respectively, and further includes:

[0045] The forward propagation process of the general basic model is modified by adding a low-rank correction term to obtain a task-adapted output; a predefined loss function is used to optimize the low-rank projection matrix, and the loss function is as follows:

[0046]

[0047] Where L is the total loss, θ4bit are the quantized general basic model parameters, The i low-rank projection matrices introduced by QLoRA, D is the instruction fine-tuning dataset, l is the task loss function, (x, y) is any sample instruction fine-tuning data in the instruction fine-tuning dataset, x is the model input, and y is the actual label; using the first instruction fine-tuning dataset, multiple rounds of iterative optimization are performed on the task adaptation output until the loss function is minimized, and the first low-rank projection matrix is output; the first low-rank projection matrix and the quantized universal basic model parameters are used as the first task model weight; and so on, the N instruction fine-tuning datasets are loaded in sequence to perform iterative optimization of the low-rank projection matrix in the task adaptation output to obtain the N task model weights of the N knowledge extraction tasks.

[0048] In the embodiment of the present application, the calculation process is first adjusted by adding a low-rank correction term during the forward propagation of the basic model. The low-rank projection matrix includes A i and B i Among them A i Used to adjust the input feature representation B i Used to adjust the output feature map. This adjustment enables the base model to dynamically adapt to the task characteristics and generate task-adapted outputs for different tasks.

[0049] Next, we predefine the loss function, where the loss function is used to optimize the low-rank projection matrix. The loss function is In this formula, L represents the total loss, which is used to measure the overall deviation between the model's predicted output and the actual target. 4bit The quantized model parameters generated for QLoRA technology reduce video memory usage by quantizing the weights of the Phi3.5-4B model to 4-bit format. The i low-rank projection matrices introduced for QLoRA are used to dynamically adjust the quantized model weights to adapt them to specific tasks. D is the instruction fine-tuning dataset, which includes the input samples of the task and the corresponding label data. l is the task loss function (such as cross-entropy loss), which is used to calculate the error between the predicted result and the actual label. (x, y) is any sample instruction fine-tuning data in the instruction fine-tuning dataset, where x is the model input, i.e., the input of the literature content, and y is the actual label (e.g., whether it is related to "research methods").

[0050] During the training process, we first load the instruction fine-tuning dataset for the first task, namely the first instruction fine-tuning dataset, which contains clear task instructions, input samples, and expected labels. For example, for the "extract research methods" task, the input samples are literature sentences, and the labels are whether they are related to research methods. In the forward propagation, the quantized basic model parameters θ 4bitAnd the initial low-rank projection matrix {A1, B1} generates the prediction result Then, based on the loss function L, the error between the predicted result and the actual label is calculated to optimize the low-rank projection matrix parameters.

[0051] The low-rank projection matrix {A1, B1} is adjusted through multiple rounds of iterative optimization. Each round of iteration will gradually reduce the value of the loss function until convergence. When the loss function of the first task converges, the optimized projection matrix {A1, B1} is extracted and compared with the quantized basic model parameters θ 4bit Combined, generate the first task model weight.

[0052] Then, load the instruction fine-tuning datasets of the remaining tasks in sequence and optimize the corresponding low-rank projection matrices according to the same steps. For example, for the "keyword extraction" task, optimize {A2, B2} to generate the second task model weights; for the Nth task, optimize {A N , B N The weight generation process for each task is completed independently, ensuring that the model weights are highly adaptable to the specific requirements of the corresponding task.

[0053] Finally, this process generates N task model weights for N knowledge extraction tasks.

[0054] Furthermore, in the method provided in the embodiment of the application, the forward propagation process of the general basic model is modified by adding a low-rank correction term to obtain the task adaptation output, and the method further includes:

[0055] Extract the i-th network layer from the general basic model, wherein the i-th network layer is any one of the multiple network layers of the general basic model; convert the original weight matrix W of the i-th network layer into i Quantized to 4-bit format, and define the low-rank projection matrix of the i-th network layer, wherein the low-rank projection matrix includes A i and B i ; The forward propagation process of the i-th network layer is modified by adding a low-rank correction term, wherein the modified forward propagation calculation formula is as follows:

[0056] h i ′=Dequantize(W i4bit )(x i )+A i B i f i (x i )=f i (x i )+Δ i ;

[0057] Among them, h i ′ is the corrected output, Dequantize(W i4bit ) is to dequantize the 4-bit weight into a floating point number, Δ i =A i B i f i (x i ) is a low-rank correction term introduced based on QLoRA; and so on, the forward propagation process of multiple network layers in the general basic model is modified to obtain the task adaptation output.

[0058] In the embodiment of the present application, the i-th network layer is first extracted from the general basic model. The network layer is any layer in the model structure, which is responsible for mapping and processing the input features. Its core parameter is the weight matrix W i This matrix usually has a higher dimension and is used to convert the input feature x i Mapped to the output feature space of this layer. The original forward propagation calculation formula of the i-th layer can be expressed as h i =f i (x i ), where h i Is the output of this layer, which means after the activation function f i Processed input feature x i Here, f i is the activation function of the i-th layer, usually a common activation function such as ReLU, Sigmoid, etc., x i Is the input vector of the i-th layer, usually the output or input data of the previous layer. In order to reduce the computational complexity and memory usage of the model, the weight matrix W i Perform 4-bit quantization to generate the quantized weight matrix W i4bit .

[0059] Although 4-bit quantization technology effectively reduces storage costs, it may cause certain information loss. For this reason, the low-rank projection matrix A is introduced. i and B i Used to compensate for the quantization weight loss. Specifically, the matrix A i and B i The dimensions are d×r and r×d respectively, where d is the hidden layer dimension and r is the low-rank dimension, which is set to a small value (such as 8 or 16) to ensure computational efficiency. The low-rank matrix adjusts and optimizes the task-specific features by recovering the information of the quantized weight matrix, and finally obtains the task-specific features through its product A. i B i To achieve the correction of the original weight.

[0060] In the forward propagation calculation, the quantized weight matrix W i4bitIt is necessary to restore it to floating point format through dequantization operation, and obtain Dequantize(W i4bit ). Then, the output of the network layer is corrected, and the corrected forward propagation calculation formula is h i ′=Dequantize(W i4bit )(x i )+A i B i f i (x i )=f i (x i )+Δ i ; Among them, h i ′ is the corrected output, Dequantize(W i4bit ) is to dequantize the 4-bit weight into a floating point number, f i (x i ) is the original output of the network layer, Δ i =A i B i f i (x i ) is a low-rank correction term introduced based on QLoRA, which is used to compensate for the accuracy loss caused by quantization.

[0061] By adjusting the low-rank correction term Δ i With the introduction of matrix A, the output of each layer can not only retain the feature representation ability of the original model, but also i and B i Dynamically adjust to meet the needs of specific tasks. Similarly, apply the above correction calculations layer by layer to all network layers of the general basic model to complete the task adaptation of the entire model.

[0062] Finally, after completing the correction of all network layers, the corrected layer outputs are integrated to obtain the task-adapted output.

[0063] Furthermore, in the method provided in the embodiment of the application, the first instruction is used to fine-tune the data set, and multiple rounds of iterative optimization are performed on the task adaptation output until the loss function is minimized and the first low-rank projection matrix is output, and the method further includes:

[0064] Maintaining the quantized general basic model parameters as constants; using the first instruction fine-tuning dataset as training data, performing multiple rounds of iterative optimization on the low-rank projection matrix introduced by the task adaptation output until the loss function is minimized, and outputting the optimized first low-rank projection matrix.

[0065] In the embodiment of the present application, the quantized general basic model parameters are first kept constant. Next, the first instruction fine-tuning dataset is used as training data. The dataset contains specific task instructions, input samples, and target outputs, and clearly defines the task requirements. For example, in the knowledge extraction task of scientific literature, the dataset contains the instruction "Extract research method keywords from the following literature", as well as the corresponding literature content as input and target keywords as output.

[0066] During the optimization process, a low-rank projection matrix A is introduced into the task adaptation output of the general basic model. i and B i , which is used to dynamically compensate for the information loss during the quantization process. These matrices are gradually optimized through fine-tuning training. The core of the optimization is to keep the model parameters after quantization as fixed values and only adjust the parameters of the low-rank matrix to ensure the efficiency and stability of the optimization process. Specifically, guided by the first instruction fine-tuning dataset, the parameters of the low-rank projection matrix are adjusted through multiple rounds of iterative training. In each round of training, the task adaptation output is calculated by combining the fixed parameters of the quantization weights and the current low-rank matrix, and compared with the target output defined in the fine-tuning dataset. The loss function (such as cross entropy or mean square error) is calculated to measure the accuracy of the output. The parameters of the low-rank matrix are gradually updated by the gradient descent method, so that the loss function gradually converges to the minimum value. When the loss function reaches a convergence state, the optimization of the low-rank matrix is completed, and the optimized first low-rank projection matrix is finally output.

[0067] Combining the optimized first low-rank projection matrix with fixed quantization weights significantly improves the model's performance in task adaptation, enabling it to produce high-precision results for specific tasks even with low resource requirements. This process ensures computational stability through fixed quantization weights while adapting to changing task requirements through dynamic adjustment of the low-rank matrix, providing efficient and accurate technical support for intelligent knowledge extraction.

[0068] Step S500: pre-calling a universal basic model, and completing the initialization of multiple task-specific models by dynamically loading the multiple task model weights into the universal basic model.

[0069] In the embodiment of the present application, a general base model is first pre-called. This model optimizes fixed weights through quantization technology and has general feature extraction capabilities. During the pre-call phase, the model's fixed weights are loaded and maintained as basic parameters, providing general computing power but not yet adapted to the specific needs of specific tasks. This step provides a stable computing foundation for the dynamic loading of subsequent task weights.

[0070] After pre-calling the general base model, multiple task-specific models are initialized by dynamically loading the weights of multiple task models. During the dynamic loading process, the task model weights to be loaded are dynamically determined based on the results of the task requirement analysis. These task model weights do not replace the fixed weights of the general base model, but rather work together with the fixed weights through a low-rank correction. Specifically, the fixed weights of the general base model are first used to extract general features, and then the dynamically loaded task model weights supplement and correct these features to meet the requirements of the specific task.

[0071] Dynamic loading of task model weights is achieved by mapping them to the various network layers of the common base model. This process allows the task model weights to work synergistically with the fixed weights of the common model during computation, preserving the fundamental properties of the common model while compensating for task-specific feature requirements through low-rank corrections. Once loaded, each layer of the common base model can dynamically adjust according to the task weights, achieving efficient adaptation to task requirements.

[0072] Finally, by pre-calling the general base model and dynamically loading the task model weights, the general base model completes the initialization of multiple task-specific models.

[0073] Step S600: Using the multiple task-specific models to extract knowledge from the target document content, and output multiple original extraction results.

[0074] In this embodiment, the target document content is first fed into multiple task-specific models, each of which performs a corresponding function for a specific knowledge extraction task. For example, one task-specific model focuses on extracting research methods from the document, while another model is responsible for extracting research background. The task-specific models perform task-specific optimization on the input data by loading dynamic weights and combining them with the fixed weights of the general base model.

[0075] During the knowledge extraction process, the fixed weights of the task-specific model are responsible for extracting common features, ensuring a comprehensive analysis of the overall semantics of the document content. Simultaneously, dynamically loaded task model weights supplement and modify these common features for specific tasks, ensuring that the output features precisely match the task requirements. For example, when performing a keyword extraction task, dynamic weights guide the model to focus on analyzing high-frequency words and their semantic context within the document content, thereby generating a high-quality keyword list.

[0076] By combining multiple task-specific models, the different contents of the document are parsed and extracted in a targeted manner. Ultimately, multiple raw extraction results are output, each corresponding to a specific task, such as a list of extracted keywords, categorized paragraph information, or structured table data.

[0077] Step S700: After formatting the multiple original extraction results to obtain multiple formatted results, the multiple formatted extraction results are output and displayed to the user.

[0078] In an embodiment of the present application, multiple original extraction results are first sorted and classified. The original extraction results may be unstructured data such as keyword lists, paragraph content, or annotation information. The goal of formatting is to standardize these data according to task requirements, such as sorting keywords by frequency, or classifying paragraph information into different topics. Then, according to predefined formatting rules, the data is converted into a clear structured form. These formats may include tables, lists, charts, or structured text. For example, the results of the keyword extraction task are displayed in a table, showing the keywords and their number of occurrences; the results of the paragraph classification task are displayed in a grouped list, with each group corresponding to a topic.

[0079] Finally, the formatted results are displayed through the user interface. Through formatting and result display, the initially extracted data is transformed into more valuable output, helping users to use the extraction results more efficiently.

[0080] In the embodiments of the present application, in summary, the embodiments of the present application have at least the following technical effects:

[0081] The present application receives a task request and target document content input by a user; extracts key information by semantically parsing the task request and obtains a semantic parsing result; uses predefined mapping rules to map the task type of the semantic parsing result and obtains multiple knowledge extraction tasks; performs model weight matching according to multiple knowledge extraction tasks and obtains multiple task model weights; pre-calls a general basic model and initializes multiple task-specific models by dynamically loading multiple task model weights on the general basic model; uses multiple task-specific models to extract knowledge from the target document content and outputs multiple original extraction results; formats the multiple original extraction results and obtains multiple formatted results, then outputs and displays multiple formatted extraction results to the user. The present invention solves the technical problems of poor adaptability and low accuracy of knowledge extraction results in the existing technology under multi-task scenarios. By combining methods such as task parsing, mapping rule matching, and dynamic loading of model weights, the general basic model and the task-specific model are combined to realize knowledge extraction and formatted display of the target document content, accurately extract key information, and achieve the technical effect of improving adaptability to multi-task scenarios and improving the accuracy of knowledge extraction results.

[0082] Example 2, based on the same inventive concept as the adaptive knowledge extraction method for scientific literature in the above embodiment, Figure 2As shown, the present application provides an adaptive knowledge extraction system for scientific and technological literature. The system and method embodiments in the present application are based on the same inventive concept. The system includes:

[0083] An information receiving unit 11 receives a task request and target document content input by a user; a semantic parsing unit 12 extracts key information by semantically parsing the task request to obtain a semantic parsing result; a task type mapping unit 13 maps the task type of the semantic parsing result using a predefined mapping rule to obtain a plurality of knowledge extraction tasks; a weight matching unit 14 performs model weight matching based on the plurality of knowledge extraction tasks to obtain a plurality of task model weights; a model initialization unit 15 pre-calls a general basic model and completes the initialization of a plurality of task-specific models by dynamically loading the plurality of task model weights into the general basic model; a knowledge extraction unit 16 extracts knowledge from the target document content using the plurality of task-specific models and outputs a plurality of original extraction results; and a formatting processing unit 17 formats the plurality of original extraction results to obtain a plurality of formatted results, and then outputs and displays the plurality of formatted extraction results to the user.

[0084] Furthermore, the system is also used to implement the following functions:

[0085] The task request is semantically analyzed by using an Agent module to extract key request information, wherein the key request information includes a query target, a query scope, and specific constraints; the key request information is organized into structured data, and the semantic parsing result is output.

[0086] Furthermore, the system is also used to implement the following functions:

[0087] According to the mapping logic of K basic knowledge types and task types, the predefined mapping rules are constructed; based on the query target in the semantic parsing result, the knowledge types are matched with the predefined mapping rules, and a knowledge type list is output; the query scope in the semantic parsing result is used to perform range screening of the knowledge type list, and a range screening list is output; the specific constraint conditions in the semantic parsing result are used to perform constraint condition screening of the range screening list, and a target list is output; and the multiple knowledge extraction tasks are generated according to the target list.

[0088] Furthermore, the system is also used to implement the following functions:

[0089] Set N numbers of training samples for N kinds of knowledge extraction tasks; construct N instruction fine-tuning data sets for the N kinds of knowledge extraction tasks with the N numbers of training samples as constraints; perform QLoRA fine-tuning training on the universal basic model based on the N instruction fine-tuning data sets to generate N task model weights for the N kinds of knowledge extraction tasks respectively; and dynamically call the multiple task model weights corresponding to the multiple knowledge extraction tasks from the N task model weights.

[0090] Furthermore, the system is also used to implement the following functions:

[0091] The forward propagation process of the general basic model is modified by adding a low-rank correction term to obtain a task-adapted output; a predefined loss function is used to optimize the low-rank projection matrix, and the loss function is as follows:

[0092]

[0093] Where L is the total loss, θ 4bit are the quantized general basic model parameters, The i low-rank projection matrices introduced by QLoRA, D is the instruction fine-tuning dataset, l is the task loss function, (x, y) is any sample instruction fine-tuning data in the instruction fine-tuning dataset, x is the model input, and y is the actual label; using the first instruction fine-tuning dataset, multiple rounds of iterative optimization are performed on the task adaptation output until the loss function is minimized, and the first low-rank projection matrix is output; the first low-rank projection matrix and the quantized universal basic model parameters are used as the first task model weight; and so on, the N instruction fine-tuning datasets are loaded in sequence to perform iterative optimization of the low-rank projection matrix in the task adaptation output to obtain the N task model weights of the N knowledge extraction tasks.

[0094] Furthermore, the system is also used to implement the following functions:

[0095] Maintaining the quantized general basic model parameters as constants; using the first instruction fine-tuning dataset as training data, performing multiple rounds of iterative optimization on the low-rank projection matrix introduced by the task adaptation output until the loss function is minimized, and outputting the optimized first low-rank projection matrix.

[0096] Furthermore, the system is also used to implement the following functions:

[0097] Extract the i-th network layer from the general basic model, wherein the i-th network layer is any one of the multiple network layers of the general basic model; convert the original weight matrix W of the i-th network layer into iQuantized to 4-bit format, and define the low-rank projection matrix of the i-th network layer, wherein the low-rank projection matrix includes A i and B i ; The forward propagation process of the i-th network layer is modified by adding a low-rank correction term, wherein the modified forward propagation calculation formula is as follows:

[0098] h i ′=Dequantize(W i4bit )(x i )+A i B i f i (x i )=f i (x i )+Δ i ;

[0099] Among them, h i ′ is the corrected output, Dequantize(W i4bit ) is to dequantize the 4-bit weight into a floating point number, Δ i =A i B i f i (x i ) is a low-rank correction term introduced based on QLoRA; and so on, the forward propagation process of multiple network layers in the general basic model is modified to obtain the task adaptation output.

[0100] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0102] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. An adaptive knowledge extraction method for scientific and technological literature, characterized by: The method comprises: Receive task requests and target document content input by users; Extract key information by semantically parsing the task request to obtain a semantic parsing result; Using predefined mapping rules to map the semantic parsing results to task types, and obtaining multiple knowledge extraction tasks; Perform model weight matching according to the multiple knowledge extraction tasks to obtain multiple task model weights; Pre-calling a universal basic model, and completing the initialization of multiple task-specific models by dynamically loading the multiple task model weights into the universal basic model; Using the multiple task-specific models to perform knowledge extraction on the target document content, and outputting multiple original extraction results; After formatting the multiple original extraction results to obtain multiple formatted results, the multiple formatted extraction results are output and displayed to the user.

2. The method for adaptive knowledge extraction from scientific and technological literature according to claim 1, wherein: Key information is extracted by semantically parsing the task request to obtain a semantic parsing result, the method comprising: Performing semantic analysis on the task request by using an Agent module to extract key request information, wherein the key request information includes a query target, a query scope, and specific constraints; The key request information is organized into structured data, and the semantic parsing result is output.

3. The method for adaptive knowledge extraction from scientific and technological literature according to claim 2, wherein: Using predefined mapping rules to map the semantic parsing results to task types to obtain multiple knowledge extraction tasks, the method includes: Constructing the predefined mapping rules according to the mapping logic between K basic knowledge types and task types; Based on the query target in the semantic parsing result, matching the knowledge type with the predefined mapping rule is performed, and a knowledge type list is output; Using the query scope in the semantic parsing result to perform range screening on the knowledge type list, and outputting a range screening list; Using the specific constraint conditions in the semantic parsing result to perform constraint condition screening of the range screening list, and outputting a target list; The plurality of knowledge extraction tasks are generated according to the target list.

4. The method for adaptive knowledge extraction from scientific and technological literature according to claim 3, wherein: Performing model weight matching according to the multiple knowledge extraction tasks to obtain multiple task model weights, the method includes: Set the number of N training samples for N knowledge extraction tasks; Constructing N instruction fine-tuning datasets for the N knowledge extraction tasks with the number of the N training samples as a constraint; Performing QLoRA fine-tuning training on the universal basic model based on the N instruction fine-tuning data sets to generate N task model weights for the N knowledge extraction tasks respectively; The plurality of task model weights corresponding to the plurality of knowledge extraction tasks are dynamically called from the N task model weights.

5. The method for adaptive knowledge extraction from scientific and technological literature according to claim 4, characterized in that: Performing QLoRA fine-tuning training on the universal basic model based on the N instruction fine-tuning data sets, and generating N task model weights for the N knowledge extraction tasks, respectively, the method comprising: Modifying the forward propagation process of the general base model by adding a low-rank correction term to obtain a task-adapted output; A predefined loss function is used to optimize the low-rank projection matrix. The loss function is as follows: Where L is the total loss, θ 4bit are the quantized general basic model parameters, are the i low-rank projection matrices introduced by QLoRA, D is the instruction fine-tuning dataset, l is the task loss function, (x, y) is any sample instruction fine-tuning data in the instruction fine-tuning dataset, x is the model input, and y is the actual label; Fine-tuning the dataset using the first instruction, performing multiple rounds of iterative optimization on the task adaptation output until the loss function is minimized, and outputting a first low-rank projection matrix; Using the first low-rank projection matrix and the quantized universal basic model parameters as first task model weights; Similarly, the N instruction fine-tuning data sets are loaded in sequence to perform iterative optimization of the low-rank projection matrix in the task adaptation output to obtain the N task model weights of the N knowledge extraction tasks.

6. The method for adaptive knowledge extraction from scientific and technological literature according to claim 5, characterized in that: Using a first instruction to fine-tune a data set, performing multiple rounds of iterative optimization on the task adaptation output until the loss function is minimized, and outputting a first low-rank projection matrix, the method comprising: Keep the quantized general basic model parameters as constant values; The first instruction fine-tuning dataset is used as training data, and the low-rank projection matrix introduced by the task adaptation output is iteratively optimized for multiple rounds until the loss function is minimized, and the optimized first low-rank projection matrix is output.

7. The method for adaptive knowledge extraction from scientific and technological literature according to claim 5, characterized in that: Modifying the forward propagation process of the general base model by adding a low-rank correction term to obtain a task-adapted output, the method comprising: Extracting an i-th network layer from the universal basic model, wherein the i-th network layer is any one of the multiple network layers of the universal basic model; The original weight matrix W of the i-th network layer i Quantized to 4-bit format, and define the low-rank projection matrix of the i-th network layer, wherein the low-rank projection matrix includes A i and B i ; The forward propagation process of the i-th network layer is modified by adding a low-rank correction term, wherein the modified forward propagation calculation formula is as follows: h i ′=Dequantize(W i4bit )(x i )+A i B i f i (x i )=f i (x i )+Δ i ; Among them, h i ′ is the corrected output, Dequantize(W i4bit ) is to dequantize the 4-bit weight into a floating point number, Δ i =A i B i f i (x i ) is a low-rank correction term introduced based on QLoRA; Similarly, the forward propagation process is modified on multiple network layers in the universal basic model to obtain the task adaptation output.

8. An adaptive knowledge extraction system for scientific and technological literature, characterized by: The system comprises: An information receiving unit, wherein the information receiving unit receives a task request and target document content input by a user; A semantic parsing unit, wherein the semantic parsing unit extracts key information by performing semantic parsing on the task request to obtain a semantic parsing result; A task type mapping unit, wherein the task type mapping unit uses a predefined mapping rule to perform task type mapping on the semantic parsing result to obtain a plurality of knowledge extraction tasks; A weight matching unit, wherein the weight matching unit performs model weight matching according to the multiple knowledge extraction tasks to obtain multiple task model weights; A model initialization unit, wherein the model initialization unit pre-calls a universal basic model and completes the initialization of multiple task-specific models by dynamically loading the multiple task model weights into the universal basic model; a knowledge extraction unit, wherein the knowledge extraction unit uses the multiple task-specific models to extract knowledge from the target document content and outputs multiple original extraction results; A formatting processing unit is configured to format the plurality of original extraction results to obtain a plurality of formatted results, and then output and display the plurality of formatted extraction results to the user.

Images