Self-adaptive knowledge extraction method and system for scientific and technical literature

Abstract

The invention discloses a scientific and technical literature-oriented adaptive knowledge extraction method and system, and relates to the technical field of knowledge extraction, and the method comprises the steps: receiving a task request and target literature content inputted by a user; performing key information extraction on the task request to obtain a semantic analysis result; performing task type mapping of the semantic analysis result to obtain a plurality of knowledge extraction tasks; performing model weight matching according to the plurality of knowledge extraction tasks to obtain a plurality of task model weights; task model weights are dynamically loaded for the general basic model, and initialization of a task special model is completed; and performing knowledge extraction on the target literature content, outputting a plurality of original extraction results, performing formatting processing to obtain a plurality of formatted results, and outputting and displaying the formatted results to a user. The technical problems of poor adaptation capability and low knowledge extraction result accuracy in a multi-task scene in the prior art are solved, and the technical effects of improving the adaptation capability to the multi-task scene and improving the knowledge extraction result accuracy are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge extraction, and in particular to an adaptive knowledge extraction method and system for scientific and technological literature. Background Art

[0002] Knowledge extraction for scientific and technological literature is an important means of processing and extracting key information from literature, and is widely used in scientific research literature analysis, academic search engines, intelligent question-answering systems, and other fields. However, due to the complex content, diverse structure, and wide range of fields involved in scientific and technological literature, traditional knowledge extraction methods often encounter problems such as poor adaptability and low accuracy of extraction results in multi-task scenarios. These problems not only affect the efficiency and accuracy of knowledge extraction, but also increase the difficulty of model training and adaptation.

[0003] At present, existing knowledge extraction methods mainly rely on models and algorithms designed for specific tasks, such as rule-based extraction methods, machine learning models, and deep learning methods. These methods have achieved certain success in specific tasks, but when faced with multi-task and cross-domain scientific and technological literature processing, they often lack good generalization ability and adaptability between tasks, resulting in the inability of the model to share knowledge and experience between multiple tasks, reducing the overall effectiveness of the system. Summary of the invention

[0004] The present application provides an adaptive knowledge extraction method and system for scientific and technological literature, which is used to solve the technical problems of poor adaptability of existing technologies in multi-task scenarios and low accuracy of knowledge extraction results.

[0005] In view of the above problems, the present application provides an adaptive knowledge extraction method and system for scientific and technological literature.

[0006] In a first aspect of the present application, a method for adaptive knowledge extraction for scientific and technological literature is provided, the method comprising:

[0007] Receive a task request and target document content input by a user; extract key information by semantically parsing the task request to obtain a semantic parsing result; use predefined mapping rules to map the semantic parsing result to a task type to obtain a plurality of knowledge extraction tasks; perform model weight matching according to the plurality of knowledge extraction tasks to obtain a plurality of task model weights; pre-call a general basic model and initialize a plurality of task-specific models by dynamically loading the plurality of task model weights into the general basic model; use the plurality of task-specific models to perform knowledge extraction on the target document content and output a plurality of original extraction results; after formatting the plurality of original extraction results to obtain a plurality of formatted results, output and display the plurality of formatted extraction results to the user.

[0008] The second aspect of the present application provides an adaptive knowledge extraction system for scientific and technological literature, the system comprising:

[0009] An information receiving unit, which receives a task request and target document content input by a user; a semantic parsing unit, which extracts key information by semantically parsing the task request to obtain a semantic parsing result; a task type mapping unit, which uses a predefined mapping rule to map the task type of the semantic parsing result to obtain a plurality of knowledge extraction tasks; a weight matching unit, which performs model weight matching according to the plurality of knowledge extraction tasks to obtain a plurality of task model weights; a model initialization unit, which pre-calls a general basic model and completes the initialization of a plurality of task-specific models by dynamically loading the plurality of task model weights into the general basic model; a knowledge extraction unit, which uses the plurality of task-specific models to extract knowledge from the target document content and outputs a plurality of original extraction results; a formatting processing unit, which outputs and displays the plurality of formatted extraction results to the user after formatting the plurality of original extraction results to obtain a plurality of formatted results.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] The present application receives a task request and target document content input by a user; extracts key information by semantically parsing the task request and obtains a semantic parsing result; uses a predefined mapping rule to map the task type of the semantic parsing result and obtains multiple knowledge extraction tasks; performs model weight matching according to multiple knowledge extraction tasks and obtains multiple task model weights; pre-calls a general basic model and dynamically loads multiple task model weights to the general basic model to complete the initialization of multiple task-specific models; uses multiple task-specific models to extract knowledge from the target document content and outputs multiple original extraction results; after formatting the multiple original extraction results and obtaining multiple formatted results, outputs and displays multiple formatted extraction results to the user. The present invention solves the technical problems of poor adaptability and low accuracy of knowledge extraction results in the prior art in multi-task scenarios, and realizes knowledge extraction and formatted display of the target document content through task parsing, mapping rule matching, dynamic loading of model weights and other methods, combined with a general basic model and a task-specific model, and accurately extracts key information, so as to improve the adaptability to multi-task scenarios and the technical effect of improving the accuracy of knowledge extraction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic diagram of a flow chart of an adaptive knowledge extraction method for scientific and technological literature provided in an embodiment of the present application;

[0014] Figure 2 A schematic diagram of the structure of an adaptive knowledge extraction system for scientific and technological literature provided in an embodiment of the present application.

[0015] Explanation of the accompanying drawings: information receiving unit 11, semantic parsing unit 12, task type mapping unit 13, weight matching unit 14, model initialization unit 15, knowledge extraction unit 16, formatting processing unit 17. DETAILED DESCRIPTION

[0016] The present application provides an adaptive knowledge extraction method and system for scientific and technological literature, aiming to solve the technical problems of poor adaptability and low accuracy of knowledge extraction results in existing technologies in multi-task scenarios. Through methods such as task analysis, mapping rule matching, and dynamic loading of model weights, combined with general basic models and task-specific models, the knowledge extraction and formatted display of the target document content can be achieved, and key information can be accurately extracted, thereby achieving the technical effect of improving the adaptability to multi-task scenarios and improving the accuracy of knowledge extraction results.

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0018] It should be noted that any variations of the terms "include" and "have" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0019] Embodiment 1, as Figure 1 As shown, the present application provides an adaptive knowledge extraction method for scientific and technological literature, the method comprising:

[0020] Step S100: receiving a task request and target document content input by a user.

[0021] In the embodiment of the present application, at the initial stage of knowledge extraction, the task request and target document content input by the user are received. The user submits the task request in natural language form, such as "extract research questions related to neural network optimization in the text", and clearly describes the target information and content range to be extracted. At the same time, the user provides the target document content as the source of scientific and technological literature data to be processed. These contents are specific text information such as the full text, paragraphs, abstracts or titles of the document.

[0022] Step S200: extract key information by semantically parsing the task request to obtain a semantic parsing result.

[0023] In the embodiment of the present application, by semantically parsing the task request, the Agent module is used to perform semantic analysis on the natural language task request input by the user, and the key information in the request is gradually extracted. This key information includes the specific content that the user wants to obtain, that is, the query target (such as "research method"), the clear query scope (such as "in this article"), and the specific constraints defined by the user (such as "related to deep learning"). After extracting this information, it is organized in a structured form to generate a semantic parsing result.

[0024] Furthermore, in the method provided in the embodiment of the application, the key information is extracted by semantically parsing the task request to obtain the semantic parsing result, and the method further includes:

[0025] The task request is semantically analyzed by using an Agent module to extract key request information, wherein the key request information includes a query target, a query scope, and specific constraints; the key request information is organized into structured data, and the semantic parsing result is output.

[0026] In an embodiment of the present application, by using the Agent module, the task request submitted by the user is first received and a semantic analysis is performed on it. During the analysis process, the Agent module extracts key information in the task request by means of semantic understanding, which includes the query target, query scope and specific constraints. The query target is the main type of information that the user wants to extract from the literature, such as "research methods" or "research questions". The query scope limits the processing boundaries of the task, such as the "full text" or "abstract" section. Specific constraints are additional restrictions added to the task, such as "related to deep learning" or "within a specific time period". After extracting these key information, the Agent module organizes them into structured data to ensure the standardized expression of each information point. Finally, the structured data is output as a semantic parsing result.

[0027] Step S300: using predefined mapping rules to perform task type mapping of the semantic parsing results to obtain multiple knowledge extraction tasks.

[0028] In an embodiment of the present application, first, a predefined mapping rule is constructed based on the mapping logic between K basic knowledge types and task types. Then, the knowledge types are matched with the mapping rules through the query targets in the semantic parsing results to generate a knowledge type list. Then, the knowledge type list is filtered in combination with the query range in the semantic parsing results, and a range filtering list is output. Finally, the range filtering list is further filtered according to specific constraints to generate a target list. Through this series of logical mapping and filtering, multiple knowledge extraction tasks are finally generated.

[0029] Furthermore, in the method provided in the embodiment of the application, the task type mapping of the semantic parsing result is performed using a predefined mapping rule to obtain multiple knowledge extraction tasks, and further includes:

[0030] According to the mapping logic between K basic knowledge types and task types, the predefined mapping rules are constructed; based on the query target in the semantic parsing result, the knowledge types are matched with the predefined mapping rules, and a knowledge type list is output; the query scope in the semantic parsing result is used to perform range screening on the knowledge type list, and a range screening list is output; the specific constraints in the semantic parsing result are used to perform constraint screening on the range screening list, and a target list is output; and the multiple knowledge extraction tasks are generated according to the target list.

[0031] In the embodiment of the present application, firstly, based on the logical relationship between K basic knowledge types (such as research questions, research objectives, research background, research methods, research results and keywords, etc.) and task types, predefined mapping rules are constructed. These mapping rules clarify the association between each knowledge type and the corresponding task. For example, the basic knowledge type "research methods" can correspond to the "method identification" task, and the "research question" can correspond to the "question sentence extraction" task.

[0032] Then, the preliminary screening of knowledge types is completed through query target matching. Using semantic rule-based matching technology, the query targets in the semantic parsing results are compared with the mapping rules one by one. For example, when the query target is "research method", the matching "method" knowledge type is extracted from the rule base to generate a preliminary list of knowledge types. This step ensures that the core requirements in the semantic parsing results are accurately identified.

[0033] The knowledge type list is then further filtered based on the query scope in the semantic parsing results. The query scope reflects the operational boundary of the user task, such as "in this article" or "in the abstract". Through rule filtering, the applicable scopes of the knowledge types are compared, and the knowledge types that meet the scope requirements are retained, thereby generating a scope filtering list. For example, when the scope is limited to "in the abstract", only the knowledge types that are suitable for execution in the abstract are retained.

[0034] Next, the scope filter list is further filtered based on the specific constraints in the semantic parsing results. Specific constraints refer to additional restrictions in the task requirements, such as "related to deep learning". In this step, the rule filtering method is used to match each task type in the filter list based on the conditional phrases in the semantic parsing results, and task types that do not meet the limited conditions are eliminated to finally generate a target list.

[0035] Finally, multiple knowledge extraction tasks are generated based on the target list. These tasks clearly define the target content, operation scope, and additional conditions, such as "extracting research methods related to deep learning from the full text" or "identifying research questions in the abstract section."

[0036] Step S400: performing model weight matching according to the multiple knowledge extraction tasks to obtain multiple task model weights.

[0037] In an embodiment of the present application, when matching model weights according to multiple knowledge extraction tasks, the corresponding number of training samples is first set for N types of knowledge extraction tasks, and N instruction fine-tuning data sets are constructed based on this as a constraint condition; then, based on these fine-tuning data sets, QLoRA fine-tuning training is performed on the general basic model to generate N task model weights respectively, and finally multiple task model weights corresponding to multiple knowledge extraction tasks are obtained.

[0038] Furthermore, in the method provided in the embodiment of the application, model weight matching is performed according to the multiple knowledge extraction tasks to obtain multiple task model weights, and further includes:

[0039] Set the number of N training samples for N kinds of knowledge extraction tasks; construct N instruction fine-tuning data sets for the N kinds of knowledge extraction tasks with the number of N training samples as a constraint; perform QLoRA fine-tuning training on the universal basic model based on the N instruction fine-tuning data sets, and respectively generate N task model weights for the N kinds of knowledge extraction tasks; dynamically call the multiple task model weights corresponding to the multiple knowledge extraction tasks from the N task model weights.

[0040] In the embodiment of the present application, the number of training samples corresponding to N kinds of knowledge extraction tasks (such as "extracting research methods" and "identifying research questions") is first set. These sample numbers are determined according to the complexity of the task and the semantic characteristics, and serve as the basic constraints for constructing the fine-tuning dataset. For example, the "research methods" task may require 50,000 samples, while the "keyword extraction" task may require 30,000 samples.

[0041] Then, based on the above number of samples, N instruction fine-tuning datasets are constructed. Each dataset is in the form of a triple, including "instructions" (clear task definition, such as "identify research methods"), "input" (sentences or paragraphs of literature content), and "output" (standardized results of task objectives). For example, for the "research methods" task, the input is "This paper proposes a new algorithm", and the output is "True / False", indicating whether it is a sentence related to research methods. In this way, the fine-tuning dataset covers the diverse expressions of the task and provides the model with clear guidance on the task objectives.

[0042] Then, the general basic model is fine-tuned by QLoRA based on N instruction fine-tuning datasets. QLoRA technology reduces the memory requirements by quantizing the model weights into 4-bit format, and introduces a low-rank adaptation matrix to optimize the model parameters, so that the model can efficiently adapt to specific tasks. During the training process, the model weights are adjusted independently for each task dataset to optimize its performance on the task, and finally N task model weights are generated. Each weight contains the fine-tuning parameters of a specific task and is dedicated to performing the corresponding knowledge extraction task.

[0043] Finally, multiple task model weights corresponding to multiple knowledge extraction tasks are dynamically called from N task model weights. For example, when the "extract research method" task needs to be processed, the "method identification task weight" is dynamically loaded from the weight library; and when the "keyword extraction" task is executed, the "keyword extraction task weight" is dynamically loaded.

[0044] Furthermore, in the method provided in the embodiment of the application, QLoRA fine-tuning training is performed on the general basic model based on the N instruction fine-tuning data sets, and N task model weights of the N knowledge extraction tasks are generated respectively, and the method also includes:

[0045] The forward propagation process of the general basic model is modified by adding a low-rank correction term to obtain a task adaptation output; a loss function is predefined, wherein the loss function is used to optimize the low-rank projection matrix, and the loss function is as follows:

[0046]

[0047] Where L is the total loss, θ4bit are the quantized general basic model parameters, The i low-rank projection matrices introduced by QLoRA, D is the instruction fine-tuning dataset, l is the task loss function, (x, y) is any sample instruction fine-tuning data in the instruction fine-tuning dataset, x is the model input, and y is the actual label; the first instruction fine-tuning dataset is used to perform multiple rounds of iterative optimization on the task adaptation output until the loss function is minimized, and the first low-rank projection matrix is ​​output; the first low-rank projection matrix and the quantized general basic model parameters are used as the first task model weight; and so on, the N instruction fine-tuning datasets are loaded in turn to perform iterative optimization of the low-rank projection matrix in the task adaptation output to obtain the N task model weights of the N knowledge extraction tasks.

[0048] In the embodiment of the present application, the calculation process is first adjusted by adding a low-rank correction term during the forward propagation process of the basic model. The low-rank projection matrix includes A i and B i Among them, A i Used to adjust the input feature representation B i Used to adjust the output feature map. Through adjustment, the basic model can be dynamically adapted to the task characteristics to generate task-adapted outputs for different tasks.

[0049] Next, we predefine the loss function, where the loss function is used to optimize the low-rank projection matrix. The loss function is In this formula, L represents the total loss, which is used to measure the overall deviation between the model's predicted output and the actual target. 4bit The quantized model parameters generated for the QLoRA technology reduce the memory usage by quantizing the weights of the Phi3.5-4B model to 4-bit format. The i low-rank projection matrices introduced by QLoRA are used to dynamically adjust the quantized model weights to adapt them to specific tasks. D is the instruction fine-tuning dataset, including the input samples of the task and the corresponding label data. l is the task loss function (such as cross entropy loss), which is used to calculate the error between the predicted result and the actual label. (x, y) is any sample instruction fine-tuning data in the instruction fine-tuning dataset, x is the model input, that is, the input of the literature content, and y is the actual label (such as whether it is related to "research methods").

[0050] During the training process, we first load the instruction fine-tuning dataset for the first task, namely the first instruction fine-tuning dataset, which contains clear task instructions, input samples, and expected labels. For example, for the task of “extracting research methods”, the input samples are literature sentences, and the labels are whether they are related to research methods. In the forward propagation, the quantized basic model parameters θ 4bit and the initial low-rank projection matrix {A1 , B 1 Generate prediction results Then, based on the loss function L, the error between the predicted result and the actual label is calculated to optimize the low-rank projection matrix parameters.

[0051] Through multiple rounds of iterative optimization, the low-rank projection matrix {A 1 , B 1} is adjusted, and each round of iteration will gradually reduce the value of the loss function until convergence. When the loss function of the first task converges, extract the optimized projection matrix {A 1 , B 1} and compare it with the quantized basic model parameters θ 4bit Combined, generate the first task model weight.

[0052] Then load the instruction fine-tuning datasets of the remaining tasks in turn, and optimize the corresponding low-rank projection matrices according to the same steps. For example, for the "keyword extraction" task, optimize {A 2 , B 2} and then generate the second task model weight; for the Nth task, optimize {A N , B N} and then generate the Nth task model weight. The weight generation process for each task is completed independently to ensure that the model weight can be highly adapted to the specific requirements of the corresponding task.

[0053] Finally, this process generates N task model weights for N knowledge extraction tasks.

[0054] Furthermore, in the method provided in the embodiment of the application, the forward propagation process of the general basic model is modified by adding a low-rank correction term to obtain the task adaptation output, and the method further includes:

[0055] Extract the i-th network layer from the general basic model, wherein the i-th network layer is any one of the multiple network layers of the general basic model; convert the original weight matrix W of the i-th network layer i Quantized to 4-bit format, and defining the low-rank projection matrix of the i-th network layer, wherein the low-rank projection matrix includes A i and B i ; The forward propagation process of the i-th network layer is modified by adding a low-rank correction term, wherein the modified forward propagation calculation formula is as follows:

[0056] h i ′=Dequantize(W i4bit )(x i )+A i B i f i (xi )=f i (x i )+Δ i ;

[0057] Among them, h i ′ is the corrected output, Dequantize(W i4bit ) is to dequantize the 4-bit weight into a floating point number, Δ i =A i B i f i (x i ) is a low-rank correction term introduced based on QLoRA; and by analogy, the forward propagation process of multiple network layers in the general basic model is modified to obtain the task adaptation output.

[0058] In the embodiment of the present application, the i-th network layer is first extracted from the general basic model. The network layer is any layer in the model structure, which is responsible for mapping and processing the input features. Its core parameter is the weight matrix W i This matrix usually has a high dimension and is used to convert the input feature x i Mapped to the output feature space of this layer. The original forward propagation calculation formula of the i-th layer can be expressed as h i =f i (x i ), where h i is the output of this layer, which means it has been activated by the function f i Processed input feature x i Here, f i is the activation function of the i-th layer, usually a common activation function such as ReLU, Sigmoid, etc., x i is the input vector of the i-th layer, usually the output or input data of the previous layer. To reduce the computational complexity and memory usage of the model, the weight matrix W i Perform 4-bit quantization to generate the quantized weight matrix W i4bit .

[0059] Although 4-bit quantization technology effectively reduces storage costs, it may cause certain information loss. For this reason, a low-rank projection matrix A is introduced. i and B i Used to compensate for the quantization weight loss. Specifically, the matrix A i and B i The dimensions of are d×r and r×d respectively, where d is the hidden layer dimension and r is the low-rank dimension, which is set to a small value (such as 8 or 16) to ensure computational efficiency. The low-rank matrix recovers the information of the quantized weight matrix, adjusts and optimizes the task-specific features, and finally obtains the result through its product A. i Bi To achieve the correction of the original weight.

[0060] In the forward propagation calculation, the quantized weight matrix W i4bit It needs to be restored to floating point format through dequantization operation to obtain Dequantize(W i4bit ). Then, the output of the network layer is corrected, and the corrected forward propagation calculation formula is h i ′=Dequantize(W i4bit )(x i )+A i B i f i (x i )=f i (x i )+Δ i ; Among them, h i ′ is the corrected output, Dequantize(W i4bit ) is to dequantize the 4-bit weight into a floating point number, f i (x i ) is the original output of the network layer, Δ i =A i B i f i (x i ) is a low-rank correction term introduced based on QLoRA, which is used to compensate for the accuracy loss caused by quantization.

[0061] By adjusting the low-rank correction term Δ i With the introduction of matrix A, the output of each layer can not only retain the feature representation ability of the original model, but also i and B i Dynamic adjustment is performed to adapt to the needs of specific tasks. Similarly, the above correction calculations are applied layer by layer to all network layers of the general basic model to complete the task adaptation of the entire model.

[0062] Finally, after completing the correction of all network layers, the corrected layer outputs are integrated to obtain the task-adapted output.

[0063] Furthermore, in the method provided in the embodiment of the application, the first instruction is used to fine-tune the data set, and multiple rounds of iterative optimization are performed on the task adaptation output until the loss function is minimized, and the first low-rank projection matrix is ​​output, and the method also includes:

[0064] The parameters of the quantized general basic model are kept as constants; the first instruction fine-tuning data set is used as training data, and multiple rounds of iterative optimization are performed on the low-rank projection matrix introduced by the task adaptation output until the loss function is minimized, and the optimized first low-rank projection matrix is ​​output.

[0065] In the embodiment of the present application, the quantized general basic model parameters are first kept constant. Next, the first instruction fine-tuning data set is used as training data, which contains specific task instructions, input samples and target outputs, and clearly defines the task requirements. For example, in the knowledge extraction task of scientific and technological literature, the data set contains the instruction "Extract research method keywords from the following literature", as well as the corresponding literature content as input and target keywords as output.

[0066] During the optimization process, a low-rank projection matrix A is introduced for the task adaptation output of the general base model. i and B i , which is used to dynamically compensate for the information loss in the quantization process. These matrices are gradually optimized through fine-tuning training. The core of the optimization is to keep the model parameters after quantization as fixed values ​​and only adjust the parameters of the low-rank matrix to ensure the efficiency and stability of the optimization process. Specifically, guided by the first instruction fine-tuning dataset, the parameters of the low-rank projection matrix are adjusted through multiple rounds of iterative training. In each round of training, the task adaptation output is calculated by combining the fixed parameters of the quantization weights and the current low-rank matrix, and it is compared with the target output defined in the fine-tuning dataset, and the loss function (such as cross entropy or mean square error) is calculated to measure the accuracy of the output. The parameters of the low-rank matrix are gradually updated by the gradient descent method, so that the loss function gradually converges to the minimum value. When the loss function reaches a convergence state, the optimization of the low-rank matrix is ​​completed, and the optimized first low-rank projection matrix is ​​finally output.

[0067] The optimized first low-rank projection matrix combined with the fixed quantization weights can significantly improve the performance of the model in task adaptation, allowing the model to output high-precision results for specific tasks with low resource requirements. This process ensures the stability of the calculation through fixed quantization weights, and adapts to the changing requirements of the task by dynamically adjusting the low-rank matrix, providing efficient and accurate technical support for intelligent knowledge extraction.

[0068] Step S500: pre-calling a universal basic model, and completing the initialization of multiple task-specific models by dynamically loading the multiple task model weights into the universal basic model.

[0069] In the embodiment of the present application, a general basic model is first pre-called, which optimizes fixed weights through quantization technology and has general feature extraction capabilities. In the pre-call stage, the fixed weights of the model are loaded and maintained as basic parameters, providing general computing capabilities, but have not yet been adapted to the specific needs of specific tasks. This step provides a stable computing basis for the dynamic loading of subsequent task weights.

[0070] After completing the pre-call of the general base model, multiple task-specific models are initialized by dynamically loading multiple task model weights. During the dynamic loading process, the task model weights that need to be loaded are dynamically determined based on the task requirement analysis results. These task model weights do not replace the fixed weights of the general base model, but work together with the fixed weights in the form of low-rank correction. Specifically, the fixed weights of the general base model are first used to extract general features, and then the dynamically loaded task model weights supplement and correct these features to meet the needs of specific tasks.

[0071] The dynamic loading of task model weights is achieved by mapping them to the various network layers of the general base model. This process enables the task model weights to work together with the fixed weights of the general model during the calculation process, which not only retains the basic characteristics of the general model, but also compensates for the task-specific feature requirements through low-rank correction terms. After loading, each layer of the general base model can be dynamically adjusted according to the task weights to achieve efficient adaptation to task requirements.

[0072] Finally, by pre-calling the general base model and dynamically loading the task model weights, the general base model completes the initialization of multiple task-specific models.

[0073] Step S600: Using the multiple task-specific models to extract knowledge from the target document content, and output multiple original extraction results.

[0074] In the embodiment of the present application, the target document content is first input into multiple task-specific models, and each task-specific model performs corresponding functions for a specific knowledge extraction task. For example, one task-specific model focuses on extracting research methods from the literature, while another model is responsible for extracting research background. The task-specific model performs task-specific optimization processing on the input data by loading dynamic weights and combining them with the fixed weights of the general basic model.

[0075] During the knowledge extraction process, the fixed weights of the task-specific model are responsible for completing the extraction of common features, ensuring a comprehensive analysis of the overall semantics of the document content. At the same time, the dynamically loaded task model weights supplement and modify the common features for specific tasks, so that the output features can accurately match the task requirements. For example, when performing a keyword extraction task, the dynamic weights guide the model to focus on analyzing high-frequency words and their semantic context in the document content, thereby generating a high-quality keyword list.

[0076] Through the collaborative work of multiple task-specific models, different contents of the document are parsed and extracted in a targeted manner. Finally, multiple raw extraction results are output, each corresponding to a specific task, such as an extracted keyword list, classified paragraph information, or structured table data.

[0077] Step S700: after formatting the multiple original extraction results to obtain multiple formatted results, output and display the multiple formatted extraction results to the user.

[0078] In an embodiment of the present application, multiple original extraction results are first sorted and classified. The original extraction results may be unstructured data such as keyword lists, paragraph content, or annotation information. The goal of formatting is to standardize these data according to task requirements, such as sorting keywords by frequency, or classifying paragraph information into different topics. Then, according to predefined formatting rules, the data is converted into a clear structured form. These formats may include tables, lists, charts, or structured text. For example, the results of the keyword extraction task are displayed in a table, showing the keywords and their number of occurrences; the results of the paragraph classification task are displayed in a grouped list, and each group corresponds to a topic.

[0079] Finally, the formatted results are displayed through the user interface. Through formatting and result display, the initially extracted data is transformed into more valuable output, helping users to use the extraction results more efficiently.

[0080] In the embodiments of the present application, in summary, the embodiments of the present application have at least the following technical effects:

[0081] The present application receives a task request and target document content input by a user; extracts key information by semantically parsing the task request and obtains a semantic parsing result; uses a predefined mapping rule to map the task type of the semantic parsing result and obtains multiple knowledge extraction tasks; performs model weight matching according to multiple knowledge extraction tasks and obtains multiple task model weights; pre-calls a general basic model and dynamically loads multiple task model weights to the general basic model to complete the initialization of multiple task-specific models; uses multiple task-specific models to extract knowledge from the target document content and outputs multiple original extraction results; after formatting the multiple original extraction results and obtaining multiple formatted results, outputs and displays multiple formatted extraction results to the user. The present invention solves the technical problems of poor adaptability and low accuracy of knowledge extraction results in the prior art in multi-task scenarios, and realizes knowledge extraction and formatted display of the target document content through task parsing, mapping rule matching, dynamic loading of model weights and other methods, combined with a general basic model and a task-specific model, and accurately extracts key information, so as to improve the adaptability to multi-task scenarios and the technical effect of improving the accuracy of knowledge extraction results.

[0082] Embodiment 2 is based on the same inventive concept as the adaptive knowledge extraction method for scientific and technological literature in the above embodiment. Figure 2As shown, the present application provides an adaptive knowledge extraction system for scientific and technological literature, and the system and method embodiments in the present application are based on the same inventive concept. The system includes:

[0083] An information receiving unit 11, the information receiving unit 11 receives a task request and a target document content input by a user; a semantic parsing unit 12, the semantic parsing unit 12 performs key information extraction by semantically parsing the task request to obtain a semantic parsing result; a task type mapping unit 13, the task type mapping unit 13 uses a predefined mapping rule to perform task type mapping of the semantic parsing result to obtain a plurality of knowledge extraction tasks; a weight matching unit 14, the weight matching unit 14 performs model weight matching according to the plurality of knowledge extraction tasks to obtain a plurality of task model weights; a model initialization unit 15, the model initialization unit 15 pre-calls a general basic model, and completes the initialization of a plurality of task-specific models by dynamically loading the plurality of task model weights into the general basic model; a knowledge extraction unit 16, the knowledge extraction unit 16 uses the plurality of task-specific models to perform knowledge extraction on the target document content to output a plurality of original extraction results; a formatting processing unit 17, the formatting processing unit 17 performs formatting processing on the plurality of original extraction results to obtain a plurality of formatted results, and then outputs and displays the plurality of formatted extraction results to the user.

[0084] Furthermore, the system is also used to implement the following functions:

[0085] The task request is semantically analyzed by using an Agent module to extract key request information, wherein the key request information includes a query target, a query scope, and specific constraints; the key request information is organized into structured data, and the semantic parsing result is output.

[0086] Furthermore, the system is also used to implement the following functions:

[0087] According to the mapping logic between K basic knowledge types and task types, the predefined mapping rules are constructed; based on the query target in the semantic parsing result, the knowledge types are matched with the predefined mapping rules, and a knowledge type list is output; the query scope in the semantic parsing result is used to perform range screening on the knowledge type list, and a range screening list is output; the specific constraints in the semantic parsing result are used to perform constraint screening on the range screening list, and a target list is output; and the multiple knowledge extraction tasks are generated according to the target list.

[0088] Furthermore, the system is also used to implement the following functions:

[0089] Set the number of N training samples for N kinds of knowledge extraction tasks; construct N instruction fine-tuning data sets for the N kinds of knowledge extraction tasks with the number of N training samples as a constraint; perform QLoRA fine-tuning training on the universal basic model based on the N instruction fine-tuning data sets, and respectively generate N task model weights for the N kinds of knowledge extraction tasks; dynamically call the multiple task model weights corresponding to the multiple knowledge extraction tasks from the N task model weights.

[0090] Furthermore, the system is also used to implement the following functions:

[0091] The forward propagation process of the general basic model is modified by adding a low-rank correction term to obtain a task adaptation output; a loss function is predefined, wherein the loss function is used to optimize the low-rank projection matrix, and the loss function is as follows:

[0092]

[0093] Where L is the total loss, θ 4bit are the quantized general basic model parameters, The i low-rank projection matrices introduced by QLoRA, D is the instruction fine-tuning dataset, l is the task loss function, (x, y) is any sample instruction fine-tuning data in the instruction fine-tuning dataset, x is the model input, and y is the actual label; the first instruction fine-tuning dataset is used to perform multiple rounds of iterative optimization on the task adaptation output until the loss function is minimized, and the first low-rank projection matrix is ​​output; the first low-rank projection matrix and the quantized general basic model parameters are used as the first task model weight; and so on, the N instruction fine-tuning datasets are loaded in turn to perform iterative optimization of the low-rank projection matrix in the task adaptation output to obtain the N task model weights of the N knowledge extraction tasks.

[0094] Furthermore, the system is also used to implement the following functions:

[0095] The parameters of the quantized general basic model are kept as constants; the first instruction fine-tuning data set is used as training data, and multiple rounds of iterative optimization are performed on the low-rank projection matrix introduced by the task adaptation output until the loss function is minimized, and the optimized first low-rank projection matrix is ​​output.

[0096] Furthermore, the system is also used to implement the following functions:

[0097] Extract the i-th network layer from the general basic model, wherein the i-th network layer is any one of the multiple network layers of the general basic model; convert the original weight matrix W of the i-th network layer iQuantized to 4-bit format, and defining the low-rank projection matrix of the i-th network layer, wherein the low-rank projection matrix includes A i and B i ; The forward propagation process of the i-th network layer is modified by adding a low-rank correction term, wherein the modified forward propagation calculation formula is as follows:

[0098] h i ′=Dequantize(W i4bit )(x i )+A i B i f i (x i )=f i (x i )+Δ i ;

[0099] Among them, h i ′ is the corrected output, Dequantize(W i4bit ) is to dequantize the 4-bit weight into a floating point number, Δ i =A i B i f i (x i ) is a low-rank correction term introduced based on QLoRA; and by analogy, the forward propagation process of multiple network layers in the general basic model is modified to obtain the task adaptation output.

[0100] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order and continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0102] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. An adaptive knowledge extraction method for scientific and technological literature, characterized in that: The method comprises: Receive task requests and target document content input by users; Extract key information by semantically parsing the task request to obtain a semantic parsing result; Using predefined mapping rules to map the semantic parsing results to task types, to obtain multiple knowledge extraction tasks; Perform model weight matching according to the multiple knowledge extraction tasks to obtain multiple task model weights; Pre-calling a universal basic model, and completing the initialization of multiple task-specific models by dynamically loading the multiple task model weights into the universal basic model; Using the multiple task-specific models to extract knowledge from the target document content, and outputting multiple original extraction results; After formatting the multiple original extraction results to obtain multiple formatted results, the multiple formatted extraction results are output and displayed to the user.

2. The method for adaptive knowledge extraction for scientific and technological literature according to claim 1, characterized in that: By performing semantic analysis on the task request to extract key information and obtain a semantic analysis result, the method includes: By using the Agent module to perform semantic analysis on the task request, key request information is extracted, wherein the key request information includes a query target, a query scope, and specific constraints; The request key information is organized into structured data, and the semantic parsing result is output.

3. The method for adaptive knowledge extraction for scientific and technological literature as claimed in claim 2, characterized in that: Using predefined mapping rules to map the semantic parsing results to task types to obtain multiple knowledge extraction tasks, the method includes: According to the mapping logic between K basic knowledge types and task types, construct the predefined mapping rules; Based on the query target in the semantic parsing result, matching the knowledge type with the predefined mapping rule is performed, and a knowledge type list is output; Using the query scope in the semantic parsing result to perform scope screening on the knowledge type list, and outputting a scope screening list; Using the specific constraint conditions in the semantic parsing result to perform constraint condition screening of the range screening list, and outputting a target list; The plurality of knowledge extraction tasks are generated according to the target list.

4. The method for adaptive knowledge extraction for scientific and technological literature as claimed in claim 3, characterized in that: Model weight matching is performed according to the multiple knowledge extraction tasks to obtain multiple task model weights, and the method includes: Set the number of N training samples for N knowledge extraction tasks; Taking the number of the N training samples as a constraint, constructing N instruction fine-tuning data sets for the N knowledge extraction tasks; Perform QLoRA fine-tuning training on the general basic model based on the N instruction fine-tuning data sets, and generate N task model weights for the N knowledge extraction tasks respectively; The plurality of task model weights corresponding to the plurality of knowledge extraction tasks are dynamically called from the N task model weights.

5. The method for adaptive knowledge extraction for scientific and technological literature as claimed in claim 4, characterized in that: Based on the N instruction fine-tuning data sets, QLoRA fine-tuning training is performed on the general basic model to generate N task model weights for the N knowledge extraction tasks respectively. The method includes: Modifying the forward propagation process of the general base model by adding a low-rank correction term to obtain a task-adapted output; A predefined loss function is used to optimize the low-rank projection matrix, and the loss function is as follows: Where L is the total loss, θ 4bit are the quantized general basic model parameters, are i low-rank projection matrices introduced by QLoRA, D is the instruction fine-tuning dataset, l is the task loss function, (x, y) is any sample instruction fine-tuning data in the instruction fine-tuning dataset, x is the model input, and y is the actual label; Using the first instruction to fine-tune the data set, performing multiple rounds of iterative optimization on the task adaptation output until the loss function is minimized, and outputting a first low-rank projection matrix; Using the first low-rank projection matrix and the quantized universal basic model parameters as first task model weights; By analogy, the N instruction fine-tuning data sets are loaded in sequence to perform iterative optimization of the low-rank projection matrix in the task adaptation output to obtain the N task model weights of the N knowledge extraction tasks.

6. The method for adaptive knowledge extraction for scientific and technological literature as claimed in claim 5, characterized in that: The first instruction is used to fine-tune the data set, and multiple rounds of iterative optimization are performed on the task adaptation output until the loss function is minimized, and a first low-rank projection matrix is ​​output. The method includes: Keep the quantized general basic model parameters as constant values; The first instruction fine-tuning data set is used as training data, and multiple rounds of iterative optimization are performed on the low-rank projection matrix introduced by the task adaptation output until the loss function is minimized, and the optimized first low-rank projection matrix is ​​output.

7. The method for adaptive knowledge extraction for scientific and technological literature as claimed in claim 5, characterized in that: The method comprises: performing computational modification on the forward propagation process of the general basic model by adding a low-rank correction term to obtain a task adaptation output. The method comprises: Extracting an i-th network layer from the general basic model, wherein the i-th network layer is any one of a plurality of network layers of the general basic model; The original weight matrix W of the i-th network layer i Quantized to 4-bit format, and defining the low-rank projection matrix of the i-th network layer, wherein the low-rank projection matrix includes A i and B i ; The forward propagation process of the i-th network layer is modified by adding a low-rank correction term, wherein the modified forward propagation calculation formula is as follows: h i ′=Dequantize(W i4bit )(x i )+A i B i f i (x i )=f i (x i )+Δ i ; Among them, h i ′ is the corrected output, Dequantize(W i4bit ) is to dequantize the 4-bit weight into a floating point number, Δ i =A i B i f i (x i ) is a low-rank correction term introduced based on QLoRA; By analogy, the forward propagation process is modified on multiple network layers in the general basic model to obtain the task adaptation output.

8. An adaptive knowledge extraction system for scientific and technological literature, characterized in that: The system comprises: An information receiving unit, wherein the information receiving unit receives a task request and target document content input by a user; A semantic parsing unit, wherein the semantic parsing unit extracts key information by performing semantic parsing on the task request to obtain a semantic parsing result; A task type mapping unit, wherein the task type mapping unit uses a predefined mapping rule to perform task type mapping on the semantic parsing result to obtain a plurality of knowledge extraction tasks; A weight matching unit, wherein the weight matching unit performs model weight matching according to the multiple knowledge extraction tasks to obtain multiple task model weights; A model initialization unit, wherein the model initialization unit pre-calls a universal basic model and completes the initialization of multiple task-specific models by dynamically loading the multiple task model weights into the universal basic model; A knowledge extraction unit, wherein the knowledge extraction unit uses the multiple task-specific models to extract knowledge from the target document content and outputs multiple original extraction results; A formatting processing unit, wherein after performing formatting processing on the multiple original extraction results to obtain multiple formatted results, the formatting processing unit outputs and displays the multiple formatted extraction results to the user.

Images