A method for predicting drug interaction relationships based on LLMs and ICLs

Abstract

The application provides a drug interaction relationship prediction method based on LLMs and ICL, belongs to the technical field of drug interaction relationship prediction, and solves the problems of high data set arrangement cost of large language models in the field of DDI related characteristic prediction, limited generalization ability in zero sample scenarios, and the need for better guidance in integrating multi-source data such as molecules, physiology and clinic to improve prediction performance, including: extracting drug data samples and making a data set; screening the drug data samples in the data set, selecting the top K most similar positive and negative sample drug pairs; constructing a prompt for the drug relationship prediction model based on context learning and the selected positive and negative sample drug pairs; mixing all drug relationship prediction models using MOE, evaluating the scores of the drug relationship prediction models, weighting and fusing according to the scores, and obtaining the final DDI prediction result.

Description

Technical Field

[0001] This invention relates to a method for predicting drug interactions based on LLMs and ICLs, belonging to the field of drug interaction prediction technology. Background Technology

[0002] Multidrug therapy, the simultaneous use of multiple medications, is common in the treatment of patients with various diseases. However, due to drug interactions, it can lead to adverse drug reactions (DDIs). DDIs account for up to 30% of all reported adverse drug reactions, significantly impacting patient safety, morbidity, mortality, and healthcare costs. Given the complexity of diseases and the limitations of monotherapy, while combination therapies have the potential to improve efficacy, they also increase the risk of unintended interactions. Therefore, accurate DDI prediction is crucial for improving treatment outcomes and minimizing adverse reactions. Although DDI research has become a major focus, identifying DDIs remains extremely challenging due to limited clinical trial resources and the rapid growth of biomedical data.

[0003] Current state-of-the-art methods for DDI prediction include traditional machine learning and deep learning methods. Deep learning methods, in particular, have achieved outstanding performance by employing techniques such as deep neural networks (DNNs), convolutional neural networks (CNNs), graph neural networks (GNNs), and self-attention mechanisms (transformers). However, these methods often perform poorly in zero-shot scenarios and have limitations in their ability to learn from large-scale, multi-source data integration.

[0004] Large language models (LLMs), such as GPT-4, Claude, LLaMA, and Mistral, have achieved remarkable success in various general tasks due to their large-scale parameter configurations, pre-training methods, and advanced neural network architectures. However, while large language models excel in general tasks, their capabilities in specialized applications remain significantly limited.

[0005] In the field of drug discovery, large language models have demonstrated significant potential in several areas, including multi-source data integration, downstream task design, and application-specific cue strategy optimization. These advances have enabled large language models to perform tasks such as molecular property prediction and molecular transformation. However, challenges remain in areas such as property prediction related to drug discovery identification (DDI), including the high cost of organizing domain-specific datasets, limited generalization ability in zero-shot scenarios, and the need for better guidance in integrating multi-source data from molecular, physiological, and clinical sources to improve predictive performance. Summary of the Invention

[0006] This invention addresses the challenges of high dataset preparation costs, limited generalization ability in zero-shot scenarios, and the need for better guidance in integrating multi-source data such as molecular, physiological, and clinical data to improve prediction performance in the field of large language models. Therefore, it proposes a drug interaction prediction method based on LLMs and ICL.

[0007] The technical solution adopted by the present invention to solve the above problems is as follows: The present invention includes the following steps:

[0008] Step 1: Extract drug data samples and create a dataset;

[0009] Step 2: Filter the drug data samples in the dataset and select the top K positive and negative drug pairs with the highest similarity;

[0010] Step 3: Based on context learning and the selected positive and negative sample drug pairs, construct a prompt for the drug relationship prediction model, where prompt is a dialog box;

[0011] Step 4: Use MOE to mix all drug relationship prediction models, evaluate the scores of the drug relationship prediction model prompt content, and perform weighted fusion based on the scores to obtain the final DDI prediction result. Here, MOE is expert mixture and DDI is adverse drug reaction.

[0012] Preferably, step 2 specifically includes:

[0013] Calculate the similarity between drug samples and select the K pairs of positive and negative samples with the highest similarity that are greater than a preset value.

[0014] The formula for calculating the similarity between Tanimoto and Tanimoto is:

[0015]

[0016] In formula (1), x is the feature vector of the first drug sample, y is the feature vector of the second drug sample, x·y is the dot product / inner product of feature vectors x and y, and x·y is obtained through Σx i y i Σx was calculated. i y i The sum of the products of the corresponding eigenvectors, ||x|| 2 Let ||x|| be the square norm of vectors x and x. 2 pass Calculations show that Let ||y|| be the sum of the squares of the elements of the eigenvector x. 2 Let ||y|| be the square norm of vectors y and ||y||. 2 pass The calculation yields ||y||2 Sim is the sum of the squares of the elements of the eigenvector y. T The value of (x,y) ranges from 0 to 1.

[0017] Preferably, the prompt of the drug relationship prediction model in step 3 follows a structured format, including an input requirements module, a prediction task module, a consideration factors module, and an example module.

[0018] The input requirement module is used to input the name of the drug and the corresponding SMILES structure. The SMILES structure is a single-line text expressing the structure of the compound.

[0019] The prediction task module is used to determine the interaction between two input drugs based on the Tanimoto similarity calculation formula. If the Tanimoto similarity is greater than the preset value, there is an interaction between the two input drugs; if the Tanimoto similarity is less than the preset value, there is no interaction between the two input drugs.

[0020] The Considerations module is used to provide the pharmacodynamics, metabolic pathways, receptor interactions, and related clinical data analysis results of the corresponding drugs;

[0021] The example module is a practical demonstration of the expected output for zero-sample and few-sample scenarios, combining the analysis results and interactions of the input drugs.

[0022] Preferably, each evaluation criterion is scored on a scale of 1 to 5, specifically including:

[0023] Scientific accuracy is used to judge whether the content in the prompt of the current drug relationship prediction model conforms to current scientific knowledge and whether there are obvious errors or logical problems. If 0-24% of the content in the prompt of the current drug relationship prediction result conforms to current scientific knowledge and logical problems, the score is 1; if only 25%-49% of the content conforms to current scientific knowledge and logical problems, the score is 2; if only 50%-74% of the content conforms to current scientific knowledge and logical problems, the score is 3; if only 75%-99% of the content conforms to current scientific knowledge and logical problems, the score is 4; if all the content conforms to current scientific knowledge and logical problems, the score is 5.

[0024] Clarity and coherence are used to judge whether the content in the prompt of the current drug relationship prediction model is logically clear and whether the language is coherent and easy to understand. If 0-24% of the content in the prompt of the current drug relationship prediction results is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 1. If only 25%-49% of the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 2. If only 50%-74% of the content conforms to current scientific knowledge and logical issues, the score is 3. If only 75%-99% of the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 4. If all the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 5.

[0025] Evidence support is used to determine whether the content in the prompt of the current drug relationship prediction model cites sufficient evidence and reasonable reasoning to support the prediction results. If 0-24% of the content in the prompt of the current drug relationship prediction model completely lacks evidence support and reasoning, the score is 4; if only 25%-49% of the content completely lacks evidence support and reasoning, the score is 3; if only 50%-74% of the content completely lacks evidence support and reasoning, the score is 2; if only 75%-99% of the content completely lacks evidence support and reasoning, the score is 1; if none of the content lacks evidence support or reasoning, the score is 5.

[0026] Relevance is used to determine whether the content in the prompt of the current drug relationship prediction model closely revolves around the prediction task and whether it contains redundant information. If 0-24% of the content in the prompt of the current drug relationship prediction model closely revolves around the prediction task and does not contain redundant information, the score is 1; if only 25%-49% of the content closely revolves around the prediction task and does not contain redundant information, the score is 2; if only 50%-74% of the content closely revolves around the prediction task and does not contain redundant information, the score is 3; if only 75%-99% of the content closely revolves around the prediction task and does not contain redundant information, the score is 4; and if all the content closely revolves around the prediction task and does not contain redundant information, the score is 5.

[0027] Preferably, the scoring of the drug relationship prediction model in step 4 specifically includes:

[0028] GPT-4 was used as the discriminator. Based on the four criteria of the discriminator, a score for each criterion was calculated. The scores of each criterion were added together to obtain the score of the current drug relationship prediction model prompt content. The four criteria of the discriminator include scientific accuracy, clarity and coherence, evidence support and relevance. Each criterion was scored on a scale of 1 to 5.

[0029] Preferably, step 4 involves weighted fusion based on the scores to obtain the final DDI prediction result, specifically including:

[0030] A weighted fusion method is used to combine the scores of the prompt content of all drug relationship prediction models. A corresponding weight is assigned to each drug relationship prediction model's prompt content score, and the score of each prompt content is multiplied by its assigned weight to obtain a weighted score. The weighted scores of all drug relationship prediction models are then summed to obtain the final DDI prediction score S. final ;

[0031] DDI final prediction score S final The calculation formula is:

[0032]

[0033] In formula (2), S final The final prediction score for DDI is given by N, where N is the total number of drug relationship prediction models, and w is the total number of models. i The weights S for each drug relationship prediction model. model The total score for each drug relationship prediction model.

[0034] The beneficial effects of this invention are:

[0035] 1. This invention enables the prediction of drug interaction relationships in zero-sample and few-sample scenarios. Compared with existing methods in few-shot scenarios, the prediction performance of this invention is significantly higher than that of existing methods.

[0036] 2. This invention provides a specific score for drug-drug interactions and performs weighted fusion of these scores, making the prediction of drug-drug interactions more accurate and helping clinical experts interpret drug-drug interactions and develop new drugs. Attached Figure Description

[0037] Figure 1 A flowchart of a drug interaction prediction method based on LLMs and ICL provided by the present invention;

[0038] Figure 2 The structural framework diagram of a drug interaction prediction method based on LLMs and ICL provided by the present invention;

[0039] Figure 3 This is a schematic diagram of the zero-sample scenario DDI prediction prompt provided by the present invention;

[0040] Figure 4 This is a schematic diagram of the DDI prediction prompt for a few-sample scenario provided by the present invention;

[0041] Figure 5 This is a schematic diagram of the discriminator prompt provided by the present invention. Detailed Implementation

[0042] Combination Figure 1-5 This implementation method is described below. In this implementation method, LLMs stands for Large Language Model, and ICL stands for Context Learning. Figure 1 and Figure 2 As shown, the steps of the drug interaction prediction method based on LLMs and ICL described in this embodiment include:

[0043] S1: Extract drug data samples and create a dataset;

[0044] This implementation uses the Luo dataset (Luo et al. A network integration approach for drug-target interaction prediction and computational drug repositioning from heterogeneous information.). The Luo dataset contains the following, as shown in Table 1, which contains a total of 10,036 drug-target interactions.

[0045] Table 1

[0046]

[0047] S2: Filter the drug data samples in the dataset and select the top K positive and negative sample drug pairs with the highest similarity;

[0048] In this embodiment, Context Learning (ICL) is a prompting paradigm applied to large language models. It enhances the capabilities of large language models by using a small number of prompts. In DDI prediction, this embodiment needs to study how to find more suitable prompt examples. To better select prompt examples, this embodiment proposes an ICL positive and negative sample selection method for DDI based on drug similarity calculation. Three widely used similarity measures include Tanimoto similarity, cosine similarity, and Dice similarity. In drug similarity calculation, Dice similarity emphasizes shared structural features, making it suitable for identifying common substructures. Cosine similarity focuses on the angular relationship of feature vectors, making it very suitable for analyzing high-dimensional molecular data. Tanimoto similarity balances shared and unique molecular features, making it particularly effective for comparing molecular fingerprints in cheminformatics. Ultimately, this embodiment selects Tanimoto similarity, which can be calculated using the following formula:

[0049]

[0050] In formula (1), x is the feature vector of the first drug sample, y is the feature vector of the second drug sample, x·y is the dot product / inner product of feature vectors x and y, and x·y is obtained through Σx i y i Σx was calculated. i y i The sum of the products of the corresponding eigenvectors, ||x|| 2 Let ||x|| be the square norm of vectors x and x. 2 pass Calculations show that Let ||y|| be the sum of the squares of the elements of the eigenvector x. 2 Let ||y|| be the square norm of vectors y and ||y||. 2 pass The calculation yields ||y|| 2 Sim is the sum of the squares of the elements of the eigenvector y. T The value of (x,y) ranges from 0 to 1.

[0051] S3: A prompt for building a drug relationship prediction model based on contextual learning and selected positive and negative sample drug pairs;

[0052] In recent advances in language modeling, Contextual Learning (ICL) has emerged as a method that enables models to learn tasks without explicit fine-tuning. ICL achieves this by providing examples in the input, allowing the model to understand the task through context and generate accurate output. Based on a selection of positive and negative sample examples, this implementation constructs cues for DDI prediction, categorized into zero-sample and few-sample scenarios.

[0053] In zero-shot scenarios, the model makes predictions solely based on its pre-trained knowledge, without relying on specific examples. This approach is suitable for predicting interactions between novel or previously unseen drug combinations. In contrast, few-shot scenarios provide a limited number of examples to help guide the model's predictions, especially when data is limited or relevant examples are scarce; the prompts follow a structured format, consisting of several key parts: input requirements, prediction task, considerations, and examples. The input requirements specify the drug names and their corresponding SMILES structures. The prediction task is to predict whether an interaction exists between two drugs, with a result of "yes" or "no." Considerations include analysis of pharmacodynamics, metabolic pathways, receptor interactions, and relevant clinical data (including FDA labels and peer-reviewed literature). Finally, the examples section provides a practical demonstration of the input format and expected prediction output to ensure clarity of model application. The prompts for zero-shot scenarios are as follows: Figure 3 As shown, the prompts for scenarios with few samples are as follows: Figure 4 As shown.

[0054] S4: Use MOE to mix all drug relationship prediction models, evaluate the scores of the drug relationship prediction models, and perform weighted fusion based on the scores to obtain the final DDI prediction result;

[0055] In this implementation, GPT-4 is used as the discriminator to evaluate the DDI predictions generated by multiple drug relationship prediction models. The discriminator evaluates the quality of the explanations provided by each drug relationship prediction model based on four key criteria: scientific accuracy, clarity and coherence, evidence support, and relevance.

[0056] This implementation method is designed with detailed prompts for zero-sample and few-sample scenarios, such as... Figure 5 As shown, in the zero-shot scenario, the prompts clearly list the evaluation criteria and instructions for GPT-4 to evaluate each prediction and explanation. In the few-shot scenario, the prompts include several examples of high-quality evaluations to help the model understand how to score. For each criterion, each explanation is scored on a scale of 1 to 5, and an overall score is given based on the evaluation. After scoring the results of all models, a weighted fusion method is used to merge the prediction results, where the score of each model is multiplied by a predetermined weight reflecting its reliability or performance, and then the weighted scores are summed to generate the final DDI prediction result.

[0057] Each evaluation criterion is scored on a scale of 1 to 5, specifically including:

[0058] Scientific accuracy is used to judge whether the content in the prompt of the current drug relationship prediction model conforms to current scientific knowledge and whether there are obvious errors or logical problems. If 0-24% of the content in the prompt of the current drug relationship prediction result conforms to current scientific knowledge and logical problems, the score is 1; if only 25%-49% of the content conforms to current scientific knowledge and logical problems, the score is 2; if only 50%-74% of the content conforms to current scientific knowledge and logical problems, the score is 3; if only 75%-99% of the content conforms to current scientific knowledge and logical problems, the score is 4; if all the content conforms to current scientific knowledge and logical problems, the score is 5.

[0059] Clarity and coherence are used to judge whether the content in the prompt of the current drug relationship prediction model is logically clear and whether the language is coherent and easy to understand. If 0-24% of the content in the prompt of the current drug relationship prediction results is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 1. If only 25%-49% of the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 2. If only 50%-74% of the content conforms to current scientific knowledge and logical issues, the score is 3. If only 75%-99% of the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 4. If all the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 5.

[0060] Evidence support is used to determine whether the content in the prompt of the current drug relationship prediction model cites sufficient evidence and reasonable reasoning to support the prediction results. If 0-24% of the content in the prompt of the current drug relationship prediction model completely lacks evidence support and reasoning, the score is 4; if only 25%-49% of the content completely lacks evidence support and reasoning, the score is 3; if only 50%-74% of the content completely lacks evidence support and reasoning, the score is 2; if only 75%-99% of the content completely lacks evidence support and reasoning, the score is 1; if none of the content lacks evidence support or reasoning, the score is 5.

[0061] Relevance is used to determine whether the content in the prompt of the current drug relationship prediction model closely revolves around the prediction task and whether it contains redundant information. If 0-24% of the content in the prompt of the current drug relationship prediction model closely revolves around the prediction task and does not contain redundant information, the score is 1; if only 25%-49% of the content closely revolves around the prediction task and does not contain redundant information, the score is 2; if only 50%-74% of the content closely revolves around the prediction task and does not contain redundant information, the score is 3; if only 75%-99% of the content closely revolves around the prediction task and does not contain redundant information, the score is 4; and if all the content closely revolves around the prediction task and does not contain redundant information, the score is 5.

[0062] After scoring the results of all models, a weighted fusion method was used to combine the scores of all drug relationship prediction models, where each model has a weight of w. i The score S output by the discriminator for model i model Confirmed. Final predicted score S final Calculate using the following formula:

[0063]

[0064] In formula (2), S final The final prediction score for DDI is given by N, where N is the total number of drug relationship prediction models, and w is the total number of models. i The weights S for each drug relationship prediction model. model The total score for each drug relationship prediction model.

[0065] This implementation uses AUC and AUPR scores as evaluation metrics. AUC stands for Area Under the ROC curve. ROC can reflect the classification ability. Its horizontal axis is the false positive rate (FPR) and the vertical axis is the true positive rate (TPR). The closer the AUC is to 1, the better the model result.

[0066] AUPR stands for Area under the Precision / Recall curve, which is the area under the PR curve. The x-axis of the PR curve represents recall, and the y-axis represents precision. The PR curve is easily affected by the sample distribution (the ratio of positive to negative samples in the training samples), therefore AUPR can be used to measure the predictive performance on imbalanced datasets. The closer the AUPR value is to 1, the better the model performance. The formulas for calculating these AUC and AUPR metrics are as follows:

[0067] Accuracy measures the proportion of correctly classified samples out of all positive samples. The formula is as follows:

[0068]

[0069] Recall measures the proportion of correctly classified positive samples out of the actual number of positive samples. The formula is as follows:

[0070]

[0071] This implementation compares the proposed method with GPT-4, GPT-3.5, Davinci-003, LLAMA 2, and LLAMA 3 in a few-shot scenario. The results are shown in Table 2.

[0072] Table 2

[0073] GPT-4 0.681 0.643 GPT-3.5 0.632 0.622 Davinci-003 0.525 0.553 LLAMA2 0.4 0.488 LLAMA3 0.631 0.658 DDI-JUDGE 0.788 0.801

[0074] As shown in Table 2, compared with other LLM methods, DDI-JUDGE has improved AUPR and AUC in this embodiment.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A method for predicting drug interaction relationships based on LLMs and ICLs, characterized in that, The steps of the drug interaction prediction method based on LLMs and ICL include: Step 1: Extract drug data samples and create a dataset; Step 2: Filter the drug data samples in the dataset and select the top K positive and negative sample drug pairs with the highest similarity; Step 3: Construct a prompt for the drug relationship prediction model based on context learning and the selected positive and negative sample drugs, where the prompt is a dialog box; The prompt for the drug relationship prediction model in step 3 follows a structured format, including an input requirements module, a prediction task module, a consideration factors module, and an example module. The input requirement module is used to input the name of the drug and the corresponding SMILES structure of the drug. The SMILES structure is a single-line text expressing the structure of the compound. The prediction task module is used to determine the interaction between two input drugs according to the Tanimoto similarity calculation formula. If the Tanimoto similarity is greater than the preset value, there is an interaction between the two input drugs. If the Tanimoto similarity is less than the preset value, there is no interaction between the two input drugs. The consideration factors module is used to provide the pharmacodynamics, metabolic pathways, receptor interactions, and related clinical data analysis results of the corresponding drugs; The example module is used to demonstrate the expected output of zero-sample and few-sample scenarios by combining the analysis results and interactions of the input drug. Step 4: Use MOE to mix all drug relationship prediction models, evaluate the scores of the drug relationship prediction model prompt content, and perform weighted fusion based on the scores to obtain the final DDI prediction result. Here, MOE is expert mixture and DDI is adverse drug reaction. Step 4, which evaluates the scoring of the drug relationship prediction model, specifically includes: GPT-4 was used as the discriminator. Based on the four criteria of the discriminator, the score of each criterion was calculated. The scores of each criterion were added together to obtain the score of the current drug relationship prediction model prompt content. The four criteria of the discriminator include scientific accuracy, clarity and coherence, evidence support and relevance. Each criterion was scored on a scale of 1 to 5. Step 4 involves weighted fusion based on the scores to obtain the final DDI prediction result, which specifically includes: A weighted fusion method is used to combine the scores of the prompt content of all drug relationship prediction models. A corresponding weight is assigned to each drug relationship prediction model's prompt content score, and the score of each prompt content is multiplied by its assigned weight to obtain a weighted score. The weighted scores of all drug relationship prediction models are then summed to obtain the final DDI prediction score. ; DDI Final Predicted Score The calculation formula is: In formula (2), The final prediction score for DDI. This represents the total number of drug relationship prediction models. Weights for each drug relationship prediction model. The total score for each drug relationship prediction model.

2. The method for predicting drug interactions based on LLMs and ICLs according to claim 1, characterized in that, Step 2 specifically includes: Calculate the similarity between drug samples and select the K pairs of positive and negative samples with the highest similarity that are greater than the preset value. The formula for calculating the similarity between Tanimoto and Tanimoto is: （1）； In formula (1), x This is the feature vector of the first drug sample. y This is the feature vector of the second drug sample. For feature vectors x and y dot product / inner product, pass Calculations show that It is the sum of the products of the corresponding eigenvectors. For vectors x The square norm of the sum, pass Calculations show that For feature vectors x The sum of the squares of the elements, For vectors y The square norm of the sum, pass Calculations show that For feature vectors y The sum of the squares of the elements, The value ranges from 0 to 1.

3. The method for predicting drug interactions based on LLMs and ICLs according to claim 1, characterized in that, Each evaluation criterion is scored on a scale of 1 to 5, specifically including: Scientific accuracy is used to judge whether the content in the prompt of the current drug relationship prediction model conforms to current scientific knowledge and whether there are obvious errors or logical problems. If 0-24% of the content in the prompt of the current drug relationship prediction result conforms to current scientific knowledge and logical problems, the score is 1; if only 25%-49% of the content conforms to current scientific knowledge and logical problems, the score is 2; if only 50%-74% of the content conforms to current scientific knowledge and logical problems, the score is 3; if only 75%-99% of the content conforms to current scientific knowledge and logical problems, the score is 4; if all the content conforms to current scientific knowledge and logical problems completely, the score is 5. Clarity and coherence are used to judge whether the content in the prompt of the current drug relationship prediction model is logically clear and whether the language is coherent and easy to understand. If 0-24% of the content in the prompt of the current drug relationship prediction results is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 1. If only 25%-49% of the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 2. If only 50%-74% of the content conforms to current scientific knowledge and logical issues, the score is 3. If only 75%-99% of the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 4. If all the content is accurately expressed, logically rigorous, clearly organized, easy to understand and without incoherence, the score is 5. Evidence support is used to determine whether the content in the prompt of the current drug relationship prediction model cites sufficient evidence and reasonable reasoning to support the prediction results. If 0-24% of the content in the prompt of the current drug relationship prediction model completely lacks evidence support and reasoning, the score is 4; if only 25%-49% of the content completely lacks evidence support and reasoning, the score is 3; if only 50%-74% of the content completely lacks evidence support and reasoning, the score is 2; if only 75%-99% of the content completely lacks evidence support and reasoning, the score is 1; if none of the content lacks evidence support or reasoning, the score is 5. The relevance score is used to determine whether the content in the prompt of the current drug relationship prediction model closely revolves around the prediction task and whether it contains redundant information. If 0-24% of the content in the prompt of the current drug relationship prediction model closely revolves around the prediction task and does not contain redundant information, the score is 1; if only 25%-49% of the content closely revolves around the prediction task and does not contain redundant information, the score is 2; if only 50%-74% of the content closely revolves around the prediction task and does not contain redundant information, the score is 3; if only 75%-99% of the content closely revolves around the prediction task and does not contain redundant information, the score is 4; and if all the content closely revolves around the prediction task and does not contain redundant information, the score is 5.

Images