Medical knowledge relationship extraction method and system based on large language model fine-tuning and retrieval enhancement generation

Abstract

The present invention discloses a method and system for extracting medical knowledge relations based on large language model fine-tuning and retrieval enhancement generation, which is applied to the field of relation extraction technology, cleans and standardizes the original training data set, and expands the original training data; uses the LoRA algorithm and RAG technology to retrieve target medical entity information from multiple professional knowledge bases; the target medical entity information is integrated into the reasoning input; comprehensively uses the fine-tuned large language model and target medical entity information, and through the design of a multi-expert system, determines the final relationship label through the majority voting method in ensemble learning. The present invention combines a large language model, retrieval enhancement technology, and LoRA fine-tuning technology to achieve accurate recognition and extraction of complex relationships in medical literature. At the same time, the present invention is also committed to optimizing the adaptability of the model so that it can show excellent extraction efficiency and accuracy in different medical sub-fields and professional data sets.

Description

Technical Field

[0001] The present invention relates to the technical field of relationship extraction, and more specifically to a medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation. Background Art

[0002] As a core technology in the field of natural language processing, relation extraction is mainly used to automatically identify semantic relationships between entities, such as cooperation, location, and affiliation, from unstructured text. This technology is of great significance for in-depth mining and understanding of large amounts of text data, especially in the rapidly developing information age, and can effectively support knowledge discovery and information retrieval systems. Medical relation extraction plays a key role in modern biomedical research and clinical decision support systems. With the rapid growth of medical research literature, manual parsing of this data has become increasingly impractical. Medical relation extraction greatly improves the efficiency and accuracy of information retrieval by automatically identifying and classifying semantic relationships between entities (such as drugs, diseases, and symptoms) in medical texts. This not only accelerates scientific discoveries, such as the mechanism of action of new drugs and the association study of disease genes, but also provides data support for clinical pathways, optimizes patient treatment plans, and thus improves the overall quality of health care. In addition, it plays an indispensable role in building medical knowledge graphs, improving the intelligence level of health information systems, and providing strong data and knowledge support for health care.

[0003] However, there are some defects in the existing technology of relation extraction in the medical field:

[0004] 1. The need for efficient knowledge extraction: The current research literature in the medical field is growing rapidly, and manual extraction and analysis of the relationships between entities has become a time-consuming and inefficient task.

[0005] 2. Lack of deep understanding: Although existing natural language processing models have made some progress in extracting entities and relationships in text, there are still major challenges in extracting complex medical texts, especially medical relationship extraction that requires deep understanding.

[0006] 3. The necessity of knowledge integration: Existing technologies lack the ability to effectively integrate information from external authoritative knowledge bases, which limits the performance of the model in relation extraction tasks, because many relation extraction requirements rely on rich background knowledge.

[0007] 4. Enhanced model adaptability: Traditional model fine-tuning methods often cannot effectively optimize for specific downstream tasks, especially in the medical field with limited data.

[0008] Therefore, how to provide an extraction method and system that can automatically, quickly and accurately identify and extract medical knowledge relationships is an urgent problem that needs to be solved by technical personnel in this field. Summary of the invention

[0009] In view of this, the present invention provides a method and system for extracting medical knowledge relations based on large language model fine-tuning and retrieval enhancement generation to solve the problems in the background technology.

[0010] In order to achieve the above object, the present invention adopts the following technical solution:

[0011] On the one hand, the present invention discloses a method for extracting medical knowledge relations based on large language model fine-tuning and retrieval enhancement generation, comprising the following specific steps:

[0012] Clean and standardize the original training data set, and expand the original training data;

[0013] The LoRA algorithm is used to introduce low-rank matrices into the large language model adjustment process for adaptive adjustment of specific tasks;

[0014] Retrieving target medical entity information from multiple professional knowledge bases using RAG; integrating the target medical entity information into reasoning input;

[0015] The fine-tuned large language model and target medical entity information are used to process the test data and infer relation labels;

[0016] Through the design of a multi-expert system, each independent model expert evaluates possible relationships and determines the final relationship label through the majority voting method in ensemble learning.

[0017] Preferably, in the above-mentioned method for extracting medical knowledge relations based on large language model fine-tuning and retrieval enhancement generation, the specific steps of expanding the original training data are as follows:

[0018] The title, summary, entity and relationship labels in the training set are input. ChatGPT-3.5Turbo summarizes the summary content with a focus on entity and relationship triplets, and pays attention to simulating the writing style of the original summary, maintaining the consistency of text length, and ensuring that the consistency and accuracy of the original data are retained after data augmentation, thereby obtaining an augmented data set consistent with the style of the original training data.

[0019] Preferably, in the above-mentioned medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation, the specific steps of using the LoRA algorithm to introduce a low-rank matrix into the large language model adjustment process for adaptive adjustment of specific tasks are as follows:

[0020] In the traditional LoRA method, ΔW is decomposed into the product of two low-rank matrices A and B, and the updated weight matrix is ​​expressed as:

[0021] W′=W+BA;

[0022] In this equation, the original weight matrix W remains unchanged, while the product of the low-dimensional matrices A and B BA provides a low-rank approximation of ΔW;

[0023] The input during fine-tuning is:

[0024] Here, N represents the number of samples in the training data set, prompt i is the hint of the i-th training sample, y i is the corresponding true relationship label; the prompt and answer are designed according to the specific medical relationship extraction task. The prompt is usually the title and abstract of the article about the medical entity, and the answer is the entity relationship predicted by the model based on these descriptions;

[0025] After receiving the input Input, during the forward propagation process, the model f generates the predicted relationship label based on the current parameters W, low-rank matrices A and B

[0026] The loss function L is calculated to evaluate the difference between the answer predicted by the model and the actual answer, thereby guiding the training and optimization of the model.

[0027] Preferably, in the above-mentioned medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation, the cross entropy loss function is used as a measurement standard, and the specific expression of the cross entropy loss function L is:

[0028]

[0029] Among them, N represents the total number of samples, y i is the true relation label of the i-th sample, is the relationship label predicted by the model for the i-th sample;

[0030] Use the back-propagation algorithm to calculate the gradient of the loss function with respect to A and B; update the parameters A and B according to the gradient and the learning rate η:

[0031]

[0032]

[0033] here, and They are the gradients of the cross loss function L with respect to A and B respectively. During the fine-tuning process, the model gradually learns how to generate accurate answers based on the input prompts, thereby improving its performance in the medical relationship extraction task.

[0034] Preferably, in the above-mentioned medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation, the specific steps of using RAG to retrieve target medical entity information from multiple professional knowledge bases are as follows:

[0035] Identify medical entities in input text, and query the medical entities in an external knowledge base;

[0036] Medical entities and retrieved text snippets are converted into vectors;

[0037] Calculate the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text segment, and sort them according to the cosine similarity;

[0038] The text segment with the highest cosine similarity is selected as the most relevant information to the medical entity and integrated into the reasoning input of the model.

[0039] Preferably, in the above-mentioned medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation, the specific steps of using RAG to retrieve target medical entity information from multiple professional knowledge bases are as follows:

[0040] Identify medical entities in input text, and query the medical entities in an external knowledge base;

[0041] During the retrieval process, BERT is used as an embedding model to generate high-dimensional feature vectors for the target medical entities and the text fragments in the retrieved knowledge base;

[0042] The process of converting medical entity E and retrieved text fragment D into vectors is expressed as follows:

[0043]

[0044]

[0045] Here, represents the feature vector of the target medical entity, and The feature vector representing the text fragment retrieved from the knowledge base; the BERT model generates an embedding vector that can represent its semantics by encoding each word in the sequence and its context; then the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text fragment is calculated; the formula for cosine similarity is:

[0046]

[0047] where · represents the vector dot product, Represents the norm of the vector, that is, the length of the vector; by comparing the cosine similarity between the feature vectors of all retrieved text fragments and the feature vector of the target medical entity, the text fragments are sorted by similarity; the text fragment with the highest cosine similarity is selected as the most relevant information to the medical entity, and the selected information fragment will be integrated into the reasoning input of the model to support subsequent medical relationship extraction tasks.

[0048] On the other hand, the present invention discloses a medical knowledge relationship extraction system based on large language model fine-tuning and retrieval enhancement generation, which is characterized by comprising:

[0049] The preprocessing and augmentation module cleans and standardizes the original training data set and expands the original training data;

[0050] The LoRA fine-tuning module uses the LoRA algorithm to introduce a low-rank matrix into the large language model adjustment process for adaptive adjustment of specific tasks;

[0051] A retrieval enhancement module, which uses RAG to retrieve target medical entity information from multiple professional knowledge bases; and integrates the target medical entity information into the reasoning input;

[0052] The reasoning and decision-making module uses a fine-tuned large language model and target medical entity information to process the test data and infer relationship labels. Through the design of a multi-expert system, each independent model expert evaluates possible relationships and determines the final relationship label through the majority voting method in ensemble learning.

[0053] Preferably, in the above-mentioned medical knowledge relationship extraction system based on large language model fine-tuning and retrieval enhancement generation, the LoRA fine-tuning module includes:

[0054] The weight matrix update unit represents the update of the weight matrix through low-rank approximation;

[0055] Prompt and answer unit, prompts and answers are designed according to specific medical relationship extraction tasks;

[0056] Forward propagation unit, generating predicted relationship labels;

[0057] The loss function unit guides the training and optimization of the model by calculating the loss function to evaluate the difference between the answer predicted by the model and the actual answer;

[0058] The back-propagation unit calculates the gradient of the loss function relative to the low-rank matrix and updates the parameter low-rank matrix according to the gradient and the learning rate.

[0059] Preferably, in the above-mentioned medical knowledge relationship extraction system based on large language model fine-tuning and retrieval enhancement generation, the retrieval enhancement module includes:

[0060] A retrieval unit, which identifies medical entities in the input text and queries the medical entities in an external knowledge base;

[0061] A vector conversion unit converts medical entities and retrieved text fragments into vectors;

[0062] A similarity ranking unit calculates the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text segment, and sorts them according to the cosine similarity;

[0063] The input unit is integrated to select the text segment with the highest cosine similarity as the most relevant information to the medical entity and integrate it into the reasoning input of the model.

[0064] It can be seen from the above technical solution that compared with the prior art, the present invention discloses a method and system for extracting medical knowledge relationships based on large language model fine-tuning and retrieval enhancement generation. First, the training data set is augmented to obtain an augmented data set consistent with the style of the original training data. Subsequently, the augmented training data is used as the total training data, and the Llama 2 large language model is fine-tuned using LoRA. Through training with a large amount of data, the model learns rich medical knowledge and how to more accurately extract the relationship between entities from medical literature, so that the Llama 2 large language model can adapt to downstream medical relationship extraction tasks. Entering the reasoning stage, in order to give the Llama 2 large language model more information about the target triple, the RAG retrieval mechanism is used to retrieve detailed information related to the target entity in multiple authoritative knowledge bases to provide the model with richer contextual prompts. The retrieved information is spliced ​​to the original test set data to form the final prompt and input into the multi-expert system. The system consists of several fine-tuned Llama 2 large language models that run independently. Each expert model makes predictions independently, and the prediction results of all expert models are integrated through an integrated learning method - majority voting method to improve the accuracy and robustness of the final output. The present invention combines the powerful understanding ability of large language models, LoRA fine-tuning and advanced retrieval enhancement technology to realize an advanced method of relation extraction in the medical field, which not only improves the model's understanding depth of medical professional texts, but also significantly improves the accuracy and efficiency of medical knowledge relation extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0066] Figure 1 This is a structural diagram of the model provided by the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] The embodiment of the present invention discloses a method for extracting medical knowledge relations based on large language model fine-tuning and retrieval enhancement generation, comprising the following specific steps:

[0069] Clean and standardize the original training data set, and expand the original training data;

[0070] The LoRA algorithm is used to introduce low-rank matrices into the large language model adjustment process for adaptive adjustment of specific tasks;

[0071] Retrieving target medical entity information from multiple professional knowledge bases using RAG; integrating the target medical entity information into reasoning input;

[0072] The fine-tuned large language model and target medical entity information are used to process the test data and infer relation labels;

[0073] Through the design of a multi-expert system, each independent model expert evaluates possible relationships and determines the final relationship label through the majority voting method in ensemble learning.

[0074] Through the above technical solution, unlike traditional relation extraction, this method combines large language models, retrieval enhancement technology, and LoRA fine-tuning technology to achieve accurate recognition and extraction of complex relationships in medical literature. At the same time, the present invention is also committed to optimizing the adaptability of the model so that it can show excellent extraction efficiency and accuracy in different medical sub-fields and professional data sets.

[0075] It is understandable that large language models such as Llama or GPT have learned rich language features through pre-training on large-scale corpora, which cover everything from basic grammar to complex semantics and deep contextual relationships. Therefore, these models have the ability to deeply understand medical texts, accurately identify medical entities in the text and their relationships, and provide high-quality feature representations. This is extremely important for extracting relationships from medical texts, because medical texts usually contain complex professional terms and semantics, which are difficult for traditional models to accurately handle.

[0076] In order to further optimize the above technical solution, the specific steps to expand the original training data are as follows:

[0077] The title, summary, entity and relationship labels in the training set are input. ChatGPT-3.5Turbo summarizes the summary content with a focus on entity and relationship triplets, and pays attention to simulating the writing style of the original summary, maintaining the consistency of text length, and ensuring that the consistency and accuracy of the original data are retained after data augmentation, thereby obtaining an augmented data set consistent with the style of the original training data.

[0078] In this way, we can ensure that the dataset contains enough variation to train a more robust model.

[0079] What you need to know is:

[0080] LoRA is a parameter-efficient method for fine-tuning large language models. This method avoids directly adjusting all model parameters during fine-tuning, and instead uses matrix decomposition techniques to represent the updates of the weight matrix through low-rank approximations.

[0081] In traditional fine-tuning, the weight matrix W of the pre-trained neural network is usually modified directly to adapt to the new task, which involves a large number of parameter updates, namely

[0082] W′=W+ΔW;

[0083] Where ΔW represents the change in weight. However, the LoRA method of the present invention proposes an assumption, called the "intrinsic rank assumption", that significant changes in the network can be captured by a lower-dimensional representation.

[0084] In other words, not all elements of ΔW are equally important, but the required adjustments can be effectively encapsulated by a smaller set of changes.

[0085] In order to further optimize the above technical solution, the LoRA algorithm is used to introduce a low-rank matrix into the large language model adjustment process for adaptive adjustment of specific tasks. The specific steps are as follows:

[0086] In the traditional LoRA method, ΔW is decomposed into the product of two low-rank matrices A and B, and the updated weight matrix is ​​expressed as:

[0087] W′=W+BA;

[0088] In this equation, the original weight matrix W remains unchanged (i.e., not updated during training), while the product of the low-dimensional matrices A and B, BA, provides a low-rank approximation of ΔW. Mathematically, if W is a d×d matrix, a traditional update of W involves d 2 With LoRA, choosing A and B to be of size d×r and r×d reduces the total number of trainable parameters to 2dr, and this number is much smaller when r < d.

[0089] The input during fine-tuning is:

[0090] Here, N represents the number of samples in the training data set, prompt i is the hint of the i-th training sample, y i is the corresponding true relationship label; the prompt and answer are designed according to the specific medical relationship extraction task. The prompt is usually the title and abstract of the article about the medical entity, and the answer is the entity relationship predicted by the model based on these descriptions;

[0091] After receiving the input Input, during the forward propagation process, the model f generates the predicted relationship label based on the current parameters W, the low-rank matrices A and B

[0092] The loss function L is calculated to evaluate the difference between the answer predicted by the model and the actual answer, thereby guiding the training and optimization of the model.

[0093] In order to further optimize the above technical solution, the cross entropy loss function is used as a measurement standard. The specific expression of the cross entropy loss function L is:

[0094]

[0095] Among them, N represents the total number of samples, y i is the true relation label of the i-th sample, is the relationship label predicted by the model for the i-th sample;

[0096] Use the back-propagation algorithm to calculate the gradient of the loss function with respect to A and B; update the parameters A and B according to the gradient and the learning rate η:

[0097]

[0098]

[0099] here, and They are the gradients of the cross loss function L with respect to A and B respectively. During the fine-tuning process, the model gradually learns how to generate accurate answers based on the input prompts, thereby improving its performance in the medical relationship extraction task.

[0100] LoRA modifies the model's weights by introducing a low-rank matrix instead of training directly on large-scale parameters. This approach can significantly reduce the number of parameters that need to be trained, making model fine-tuning more efficient. And during the fine-tuning process, most of the parameters of the pre-trained model remain unchanged, which helps the model retain the rich knowledge acquired in the pre-training stage. Therefore, the fine-tuned model is able to use this knowledge to improve the performance of new tasks.

[0101] By introducing low-rank matrices during fine-tuning, LoRA not only reduces the number of parameters that need to be adjusted, but also maintains the performance of the model and the knowledge gained during the pre-training phase. This approach effectively solves the overfitting problem caused by large models with many parameters and reduces the computational burden during training.

[0102] More importantly, LoRA allows the model to be optimized for specific medical relationship extraction tasks while retaining the advantages of pre-training, improving the flexibility, efficiency and accuracy of the model in practical applications.

[0103] The combination of large language models and LoRA fine-tuning provides a powerful technical solution for medical relationship extraction. It not only improves the accuracy of automatically identifying and classifying complex relationships from medical texts, but also optimizes the practicality of the model, making it run efficiently even in resource-limited environments, greatly promoting the development of automated processing of medical literature and medical knowledge mining.

[0104] It should be understood that RAG technology enhances the predictive power of the model by retrieving relevant external information during reasoning. By retrieving multiple professional knowledge bases, rich information related to medical entities is obtained. RAG technology combines the two methods of retrieval and generation, effectively solving the limitations of traditional generative models in dealing with complex tasks.

[0105] RAG combines traditional language models and information retrieval mechanisms. When the system needs to perform relational reasoning on specific medical entities, it not only relies on the knowledge learned within the model, but also queries external knowledge bases, such as Wikipedia, gene databases, Medical Subject Headings (MeSH) and other authoritative databases to find supplementary information about the mentioned entities and enhance the model's reasoning ability.

[0106] In order to further optimize the above technical solution, the specific steps of using RAG to retrieve target medical entity information from multiple professional knowledge bases are as follows:

[0107] Identify medical entities in input text, and query the medical entities in an external knowledge base;

[0108] Medical entities and retrieved text snippets are converted into vectors;

[0109] Calculate the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text segment, and sort them according to the cosine similarity;

[0110] The text segment with the highest cosine similarity is selected as the most relevant information to the medical entity and integrated into the reasoning input of the model.

[0111] Specifically, the specific steps of using RAG to retrieve target medical entity information from multiple professional knowledge bases are as follows:

[0112] Medical entities are identified in the input text, and then the system queries these entities in external knowledge bases, which contain rich medical information such as disease descriptions, drug mechanisms of action, biomarkers, etc.

[0113] During the retrieval process, the system uses BERT as an embedding model to generate high-dimensional feature vectors for target medical entities and text fragments in the retrieved knowledge base.

[0114] The process of converting medical entity E and retrieved text fragment D into vectors is expressed as follows:

[0115]

[0116]

[0117] Here, represents the feature vector of the target medical entity, and The feature vector representing the text fragment retrieved from the knowledge base; the BERT model generates an embedding vector that can represent its semantics by encoding each word in the sequence and its context; then the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text fragment is calculated; the formula for cosine similarity is:

[0118]

[0119] where · represents the vector dot product, Represents the norm of the vector, that is, the length of the vector; by comparing the cosine similarity between the feature vectors of all retrieved text fragments and the feature vector of the target medical entity, the text fragments are sorted by similarity; the text fragment with the highest cosine similarity is selected as the most relevant information to the medical entity, and the selected information fragment will be integrated into the reasoning input of the model to support subsequent medical relationship extraction tasks.

[0120] Furthermore, retrieval-augmented generation plays a key role in improving the model’s ability to understand complex medical terms and relationships.

[0121] RAG combines the strengths of large language models and information retrieval systems, allowing the model to dynamically retrieve relevant information from a wide range of external knowledge bases before generating answers.

[0122] The retrieved information will be integrated into the model's prediction process, allowing the model to utilize this rich contextual information to generate more accurate and detailed outputs.

[0123] This approach significantly expands the knowledge scope and adaptability of the model, making it more efficient and accurate in dealing with specialized and in-depth medical problems, because the model not only relies on its internally learned knowledge, but also dynamically utilizes the latest and relevant external data.

[0124] Another embodiment of the present invention discloses a medical knowledge relationship extraction system based on large language model fine-tuning and retrieval enhancement generation, including:

[0125] The preprocessing and augmentation module cleans and standardizes the original training data set and expands the original training data;

[0126] The LoRA fine-tuning module uses the LoRA algorithm to introduce a low-rank matrix into the large language model adjustment process for adaptive adjustment of specific tasks;

[0127] A retrieval enhancement module, using RAG to retrieve target medical entity information from multiple professional knowledge bases; the target medical entity information is integrated into the reasoning input;

[0128] The reasoning and decision-making module uses a fine-tuned large language model and target medical entity information to process the test data and infer relationship labels. Through the design of a multi-expert system, each independent model expert evaluates possible relationships and determines the final relationship label through the majority voting method in ensemble learning.

[0129] In order to further optimize the above technical solutions, the LoRA fine-tuning module includes:

[0130] The weight matrix update unit represents the update of the weight matrix through low-rank approximation;

[0131] Prompt and answer unit, prompts and answers are designed according to specific medical relationship extraction tasks;

[0132] Forward propagation unit, generating predicted relationship labels;

[0133] The loss function unit guides the training and optimization of the model by calculating the loss function to evaluate the difference between the answer predicted by the model and the actual answer;

[0134] The back-propagation unit calculates the gradient of the loss function relative to the low-rank matrix and updates the parameter low-rank matrix according to the gradient and the learning rate.

[0135] In order to further optimize the above technical solution, the search enhancement module includes:

[0136] A retrieval unit, which identifies medical entities in the input text and queries the medical entities in an external knowledge base;

[0137] A vector conversion unit converts medical entities and retrieved text fragments into vectors;

[0138] A similarity ranking unit calculates the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text segment, and sorts them according to the cosine similarity;

[0139] The input unit is integrated to select the text segment with the highest cosine similarity as the most relevant information to the medical entity and integrate it into the reasoning input of the model.

[0140] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0141] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation, characterized in that: The specific steps include: Clean and standardize the original training data set, and expand the original training data; The LoRA algorithm is used to introduce a low-rank matrix into the large language model adjustment process to perform adaptive adjustment of the task. The specific steps are as follows: In the traditional LoRA method, ΔW is decomposed into the product of two low-rank matrices A and B, and the updated weight matrix is ​​expressed as: W'=W+BA; In this equation, the original weight matrix W remains unchanged, while the product of the low-dimensional matrices A and B BA provides a low-rank approximation of ΔW; The input during fine-tuning is: Here, N represents the number of samples in the training data set, prompt i is the hint of the i-th training sample, y i is the corresponding true relationship label; the prompt and answer are designed according to the medical relationship extraction task. The prompt is usually the title and abstract of the article about the medical entity, and the answer is the entity relationship predicted by the model based on these descriptions; After receiving the input Input, during the forward propagation process, the model f generates the predicted relationship label based on the current parameters W, the low-rank matrices A and B The loss function L is calculated to evaluate the difference between the model's predicted answer and the actual answer, thereby guiding the model's training and optimization; The cross entropy loss function is used as the measurement standard. The specific expression of the cross entropy loss function L is: Among them, N represents the total number of samples, y i is the true relation label of the i-th sample, is the relationship label predicted by the model for the i-th sample; Use the back-propagation algorithm to calculate the gradient of the loss function with respect to A and B; update the parameters A and B according to the gradient and the learning rate η: here, and They are the gradients of the cross loss function L with respect to A and B respectively; Retrieving target medical entity information from multiple professional knowledge bases using RAG; integrating the target medical entity information into reasoning input; The fine-tuned large language model and target medical entity information are used to process the test data and infer relation labels; Through the design of a multi-expert system, each independent model expert evaluates possible relationships and determines the final relationship label through the majority voting method in ensemble learning.

2. The medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation according to claim 1 is characterized in that: The specific steps to expand the original training data are as follows: The title, summary, entity and relationship labels in the training set are input. ChatGPT-3.5Turbo summarizes the summary content with a focus on entity and relationship triplets, and pays attention to simulating the writing style of the original summary, maintaining the consistency of text length, and ensuring that the consistency and accuracy of the original data are retained after data augmentation, thereby obtaining an augmented data set consistent with the style of the original training data.

3. The medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation according to claim 1 is characterized in that: The specific steps of using RAG to retrieve target medical entity information from multiple professional knowledge bases are as follows: Identify medical entities in input text, and query the medical entities in an external knowledge base; Medical entities and retrieved text snippets are converted into vectors; Calculate the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text segment, and sort them according to the cosine similarity; The text segment with the highest cosine similarity is selected as the most relevant information to the medical entity and integrated into the reasoning input of the model.

4. The medical knowledge relationship extraction method based on large language model fine-tuning and retrieval enhancement generation according to claim 3 is characterized in that: The specific steps of using RAG to retrieve target medical entity information from multiple professional knowledge bases are as follows: Identify medical entities in input text, and query the medical entities in an external knowledge base; During the retrieval process, BERT is used as an embedding model to generate high-dimensional feature vectors for the target medical entities and the text fragments in the retrieved knowledge base; The process of converting medical entity E and retrieved text fragment D into vectors is expressed as follows: Here, represents the feature vector of the target medical entity, and The feature vector representing the text fragment retrieved from the knowledge base; the BERT model generates an embedding vector that can represent its semantics by encoding each word in the sequence and its context; then the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text fragment is calculated; the formula for cosine similarity is: where · represents the vector dot product, represents the norm of the vector, that is, the length of the vector; by comparing the cosine similarity between the feature vectors of all retrieved text fragments and the feature vectors of the target medical entity, the text fragments are sorted according to the similarity; The text segment with the highest cosine similarity is selected as the information most relevant to the target medical entity. The selected information segment will be integrated into the reasoning input of the model to support subsequent medical relationship extraction tasks.

5. A medical knowledge relationship extraction system based on large language model fine-tuning and retrieval enhancement generation, characterized by: include: The preprocessing and augmentation module cleans and standardizes the original training data set and expands the original training data; The LoRA fine-tuning module uses the LoRA algorithm to introduce a low-rank matrix into the large language model adjustment process to perform adaptive adjustments to the task; LoRA fine-tuning module includes: The weight matrix update unit represents the update of the weight matrix through low-rank approximation; Prompt and answer unit, prompt and answer are designed according to the medical relationship extraction task; Forward propagation unit, generating predicted relationship labels; The loss function unit guides the training and optimization of the model by calculating the loss function to evaluate the difference between the answer predicted by the model and the actual answer; Back propagation unit, calculates the gradient of the loss function relative to the low-rank matrix and updates the parameter low-rank matrix according to the gradient and learning rate; A retrieval enhancement module, which uses RAG to retrieve target medical entity information from multiple professional knowledge bases; and integrates the target medical entity information into the reasoning input; The reasoning and decision-making module uses a fine-tuned large language model and target medical entity information to process the test data and infer relationship labels. Through the design of a multi-expert system, each independent model expert evaluates possible relationships and determines the final relationship label through the majority voting method in ensemble learning.

6. The medical knowledge relationship extraction system based on large language model fine-tuning and retrieval enhancement generation according to claim 5 is characterized in that: The search enhancement modules include: A retrieval unit, which identifies medical entities in the input text and queries the medical entities in an external knowledge base; A vector conversion unit converts medical entities and retrieved text fragments into vectors; A similarity ranking unit calculates the cosine similarity between the feature vector of the medical entity and the feature vector of each retrieved text segment, and sorts them according to the cosine similarity; The input integration unit selects the text segment with the highest cosine similarity as the most relevant information to the medical entity and integrates it into the reasoning input of the model.

Images