Personalized pharmaceutical composition recommendation method and system using large language model

Abstract

The invention relates to a personalized drug combination recommendation method and system using a large language model, and the method comprises the steps: 1, collecting data from an electronic health record of a patient, including diagnosis, operation and drug information, and constructing the collected data into a data set; 2, encoding the data set to obtain encoded data; step 3, constructing a fixed prompt template, and putting the encoded data into the constructed fixed prompt template to obtain structured input; 4, constructing a recommendation model, sending the structured input into the recommendation model, introducing soft prompt embedding with the length of n in front of an embedding layer of the recommendation model, and training the recommendation model; 5, optimizing the parameters of the recommendation model by adopting a fine tuning technology, and repeatedly iterating to obtain a trained recommendation model; and performing drug recommendation by using the trained recommendation model. The method improves the accuracy of drug combination recommendation, provides an efficient solution, and adapts to different scenes and requirements.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and medical health technology, and in particular relates to a personalized drug combination recommendation method and system using a large language model. Background Art

[0002] Drug combination recommendations aim to provide patients with a set of medications to treat multiple conditions. This process relies on physician expertise and experience. Especially for patients with multiple conditions, physicians must not only select the appropriate medication for each condition but also mitigate potential drug-drug interactions (DDIs), which undoubtedly complicates the recommendation process. Therefore, developing an accurate and safe drug recommendation system that can assist physicians in decision-making is crucial.

[0003] Due to their clinical value, drug combination recommendations have attracted increasing research interest, and a series of deep learning-based drug combination recommendation methods have been proposed. These methods can be mainly divided into two categories: instance-based models and longitudinal models. However, these methods rely on complex frameworks, which complicates model design and reduces generalization ability, resulting in poor performance on large datasets. In addition, their limited scalability makes them unsuitable for processing diverse data and meeting the needs of various tasks. The emergence of large language models (LLMs) provides an opportunity to enhance existing drug recommendation systems. Through highly integrated models, they reduce the reliance on complex frameworks, improve the ease of use and efficiency of the models, and facilitate medical personnel to quickly use and understand their decision logic. Therefore, LLM-based drug recommendation methods can simplify traditional processes and improve their efficiency, thereby improving the quality of medical services. Some pioneering works have taken initial steps to integrate large language models with recommendation systems. However, their direct application to drug recommendation tasks is hindered by two important challenges.

[0004] First, traditional encoding methods lack key medical semantic information and mislead models into establishing incorrect semantic relationships, thereby reducing their robustness. Traditional drug combination recommendation methods convert datasets into corresponding identifier matrices as model input. However, they ignore the key medical semantic information in electronic health records (EHRs). Therefore, we need to assign an LLM-compatible index to each drug term identifier, ensuring its uniqueness and compatibility with natural language. Traditional encoding methods may mislead models by creating false associations between unrelated drug terms. For example, the drug terms "2048" and "2049" may be completely unrelated, but their shared prefix "20" may cause the model to incorrectly establish semantic associations between these drug terms.

[0005] The second challenge is that hard hints cannot meet the precision requirements of medicine. Most existing LLM-based recommendation methods rely on predefined fixed templates. In the LLM-based drug combination recommendation task, a common practice is to integrate patient visit information into these fixed templates and then use them as input to the LLM to generate recommendations. However, designing such templates is time-consuming and labor-intensive, and they fail to adapt to diverse datasets and provide accurate recommendations. In the specific field of drug combination recommendation, research on hint construction remains relatively scarce, further increasing the difficulty of creating effective and adaptable templates. Summary of the Invention

[0006] In order to solve the problems existing in the above-mentioned background technology, the present invention provides a personalized drug combination recommendation method and system using a large language model, which makes accurate drug combination recommendations possible, and also provides a universal and efficient solution to adapt to different scenarios and needs. Specifically, the present invention first encodes diagnostic terms and surgical terms through a coding strategy based on name indexing, encodes drug terms through a coding strategy combining collaborative indexing and name indexing, and creates medical term identifiers (IDs) compatible with LLMs. Then, a hybrid fine-tuning strategy is used to dynamically adjust the prompt template to adapt to different input features, thereby implementing a personalized prompt strategy. Finally, the LLM is adjusted by using a new output layer and an improved loss function to generate more accurate recommendation results.

[0007] Explanation of terms: LLM: Large Language Model (LLM), also known as a large language model, is an AI model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more. LLMs are characterized by their large size, containing billions of parameters, which helps them learn complex patterns in language data. These models are often based on deep learning architectures such as transformers, which contributes to their impressive performance on various NLP tasks.

[0008] The technical solutions of the present invention are as follows: A first aspect of the present invention provides a method for recommending personalized drug combinations using a large language model, comprising: Step 1: Collect data from patients’ electronic health records (EHRs), including diagnosis, surgery, and medication information, and construct a dataset to facilitate model training and evaluation. Step 2: Encode the data set to obtain the encoded data; Step 3: Build fixed prompt templates. These templates serve as a bridge for the model to understand patient data and help the model capture information more accurately. The encoded data is placed into the built fixed prompt templates to obtain structured input, which helps the model process and learn data more efficiently. Step 4: Build a recommendation model, feed the structured input into the recommendation model, introduce a soft hint embedding of length n before the embedding layer of the recommendation model, and train the recommendation model; Step 5: Use fine-tuning technology to optimize the recommendation model parameters, and iterate repeatedly to obtain a trained recommendation model; use the trained recommendation model to make drug recommendations.

[0009] Preferably, according to the present invention, encoding a data set to obtain encoded data includes: The diagnostic and surgical information in the dataset is encoded using a name-based indexing strategy. Collaborative indexing relies primarily on the similarity of patient medical data, but it does not fully utilize the powerful semantic understanding capabilities of LLMs. To address this deficiency, a name index is introduced. Using a standard mapping dictionary in the medical field, the identifiers of diagnostic and surgical information (ICD, ATC4) are converted into corresponding natural language names (for example, "A02A" is converted to "antacids"). This approach is intuitive and easy to understand because it directly uses human-readable names, helping the model capture the semantic relationships between drug terms. The encoded diagnostic and surgical terms are then obtained. The drug information in the dataset was encoded using a strategy based on a combination of collaborative indexing and name indexing; Among them, the coding strategy based on collaborative indexing utilizes collaborative information (interrelated and mutually exclusive information about diagnosis, surgery, and medication) in patients' electronic health records (EHRs) and adopts spectral clustering based on spectral matrix factorization (SMF) to generate a drug term index; it includes: Construct a graph G based on the dataset, and let G = (V, E) be the co-occurrence graph, where V represents the node set of drug terms in the electronic health record, that is, the drug information, and E represents the edge set. ij ∈E represents the co-occurrence of drug term i and drug term j, and the edge weight W ij Represents the frequency of co-occurrence of i and j. The adjacency matrix A corresponding to the graph G represents the similarity between drug terms. Based on the co-occurrence frequency, the Laplace matrix L obtained by the adjacency matrix A is factorized to achieve spectral clustering, as shown below: ; Among them, L ij A represents the Laplacian matrix of drug term i and drug term j, ij Represents the adjacency matrix of drug term i and drug term j; in the spectral clustering process, solve the Laplace matrix L The eigenvalues and eigenvectors of these eigenvectors map the data points to a new feature space (usually called "spectral space"), in which the similarities between data points are more easily captured, thereby grouping the drug term nodes into different clusters, so that drug terms with more co-occurrence similarities are grouped into the same cluster; by recursively applying the spectral clustering process within the large cluster, each cluster is further subdivided into finer-grained clusters, thereby forming a hierarchical cluster structure; two key parameters are used to control the recursive clustering process: (1) N: the number of clusters generated at each level; (2) K: the maximum number of drug terms allowed in the final cluster, which serves as the stopping condition of the recursive process; when the number of drug terms contained in a cluster does not exceed K, it will not be further divided; The final result is a hierarchical tree structure, in which each non-leaf node represents a cluster created at the corresponding level, and each leaf node represents a drug term in the corresponding final cluster. The index of each drug term is obtained by concatenating the tags of the non-leaf ancestor nodes with the tags of its own leaf nodes. This indexing method means that the more frequently two terms appear, the more tags they share, thereby effectively utilizing the collaborative information in the historical sequence; Based on the strategy of combining collaborative indexing and name indexing, the two indexes are spliced together to obtain the encoded drug term ID; this indexing method not only contains the position information of the drug term in the user behavior sequence, but also incorporates its semantic information, enabling the model to understand and process the data more comprehensively; in this way, hybrid indexing enhances the model's ability to identify complex relationships between drug terms, thereby providing more accurate results in various applications.

[0010] Build fixed prompt templates. These templates serve as a bridge for the model to understand patient data and help the model capture information more accurately. The encoded data is placed into the built fixed prompt templates to obtain structured input, which helps the model process and learn data more efficiently. This includes: A prompt template T is designed to derive the language representation P (z) of the patient's electronic health record (HER), so that the LLM can understand the patient's health status. The prompt template T is filled with personalized information such as user ID and drug term ID. The construction method of the prompt template is as follows: "Patient <PATIENT ID> Total <VISIT NUM> Intensive care unit (ICU) visit records; at the first visit, the patient's diagnosis was: <DIAG CODE>… <DIAG CODE> ; The treatment procedures performed are: <PROC CODE >……<PROC CODE> ; The prescribed medications are: <MED CODE> …<MED CODE> ; At the second visit...; In this visit, the patient's diagnosis was: <DIAG CODE> …<DIAG CODE> ; The treatment procedures performed are: <PROC CODE> …<PROC CODE> ;Next, the patient should be prescribed the following medications:"; Use the electronic health record data and the encoded data to fill in the above template. The first segment represents the patient's history V=[V1,V2,…,V T ],in,"<PATIENT_ID> " represents the unique identifier of the patient,<VISIT_NUM> "refers to the number of hospital visits a patient has,"<DIAG_CODE> ","<PROC_CODE> "and"<MED_CODE> " represents the coded data, which are the coded diagnostic terms, surgical terms, and drug term IDs. The format of each visit is the same as the first visit. Finally, a question is asked: What medications are needed after this visit? After filling in, a structured input is obtained.

[0011] Preferably, according to the present invention, a recommendation model is constructed, structured input is fed into the recommendation model, and a soft hint embedding of length n is introduced before the embedding layer of the recommendation model to train the recommendation model; the method includes: Recommended models include the improved LLM model (LLM model with a new output layer and improved loss function); The improved LLM model includes: embedding layer, Transformer decoder layer, layer normalization, and linear layer; The Transformer decoder layer includes: attention mechanism, MLP (multi-layer perceptron), layer normalization; In order to increase the flexibility of prompts, a soft personalized medical prompt adjustment method is adopted to introduce learnable parameters into the input embedding space of the recommendation model. These parameters can be optimized according to conditions such as task type and dataset. The core idea is to add a learnable projection layer to the model input layer to map the original input to the semantic space represented by the prompt information. This design enables the prompt information to better adapt to task requirements. Including, the structured input obtained in step 3 is first converted into n tags {x1, x2, ..., x n}, the continuous text string is divided into discrete units for computer processing, and embedded through the improved LLM model embedding layer to capture the semantic information and contextual relationship of the text, forming a matrix X e ∈R n×e , where e represents the dimension of the embedding space; Introduce a trainable hint parameter P of length p e ∈R p×e , represents the embedding of the soft prompt, and is combined with the embedded input matrix X e Splicing to form a new input matrix [P e ;X e ]∈R (p+n)×e , where [P e ; X e ] indicates P e and X e The row-wise splicing operation uses the spliced matrix as the model input and is processed through the encoder-decoder structure. Soft hints, as virtual embedding tags, can be optimized through training to help the model better adapt to specific tasks or domains without changing the original model parameters. The input matrix [P e ;X e ] are converted into vector representations; these vectors pass through 24 Transformer decoder layers in sequence, and are layer-normalized before and after the self-attention mechanism and MLP processing in the Transformer decoder layer to ensure the stability and performance of the model; layer normalization is also performed at the end of each Transformer decoder layer to further optimize the performance of the model; finally, the output of all Transformer decoder layers passes through a linear layer to generate the final classification score.

[0012] According to the preferred embodiment of the present invention, a linear layer with sigmoid is used to output the probability of each drug, as shown below: ; in, , is the predicted medication probability and hidden state of the last Transformer decoder layer in the improved LLM model, R represents a real number matrix, |M| represents the total number of all drugs, represents the dimension of the hidden state, is a learnable weight matrix, Represents the sigmoid function. For the final recommendation, set a threshold γ. When y k >γ, drug k will be included in the prescription drug set, where y k Represents the predicted medication probability of the Kth drug.

[0013] According to the present invention, a supervised fine-tuning (SFT) strategy is preferably used in the recommendation model training process, that is, labeled data is used to adjust the improved LLM model to make it more suitable for the specific task of drug combination recommendation. At the same time, in order to make the model more accurately adapt to the output layer of the modified LLM, the loss function is calculated by comparing the difference between the predicted output of the recommendation model and the actual target output. The loss function is set to: ; Where N is the number of drug categories, y (i) is the one-hot encoding of the true label, is the one-hot encoding of the recommendation model prediction.

[0014] According to the preferred embodiment of the present invention, fine-tuning technology is used to optimize the recommendation model parameters, and a trained recommendation model is obtained through repeated iterations; the trained recommendation model is used to make drug recommendations; Figure 3 Shown, including: To further reduce the cost of fine-tuning after introducing the soft personalized medical prompt adjustment method, the Low-Rank Adaptation of Large Language Models (LoRA) technology is used in the recommendation model training process. Low-Rank Adaptation technology updates the recommendation model weights by decomposing them into low-rank components. In this way, only the weights of the task-related subspace are adjusted, as shown below: ; Among them, W new is the updated weight, W oldis the original weight, ∆W is the low-rank update matrix; in this way, the update range of the model parameters is limited, thereby reducing the consumption of computing resources; the parameter update matrix ∆W of each layer is decomposed into two low-rank matrices: ; Where A ∈ R r×k and B ∈ R d×r are two low-rank matrices; generally speaking, r ≪ min(d , k), d represents the dimension of the input feature, k represents the dimension of the output feature, and r represents the rank of the low-rank matrix; let Represents a set of trainable matrices, where Ai and Bi are the low-rank matrices A and B of the i-th layer, respectively, and L is the number of layers of the low-rank matrix. represents the parameter set of all low-rank matrices; During the training process, the model first performs forward propagation to calculate the loss between the predicted output and the true label; then, the backpropagation algorithm calculates the partial derivative of the loss function with respect to each parameter through the chain rule. These partial derivatives indicate the contribution of the parameter to the loss; finally, using this gradient information, the model parameters are updated through the optimization algorithm (gradient descent) to reduce the loss and improve the model performance; the training is iterated continuously until the model converges to a satisfactory performance level, resulting in a trained recommendation model.

[0015] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a method for recommending personalized drug combinations using a large language model are implemented.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for recommending personalized drug combinations using a large language model.

[0017] A second aspect of the present invention provides a personalized drug combination recommendation system using a large language model, comprising: The data collection module is configured to: collect data from electronic health records, including diagnosis information, surgery information, and medication information, and construct the collected data into a dataset; The encoding module is configured to: encode the data set to obtain encoded data; The prompt template module is configured to: construct a fixed prompt template and put the encoded data into the constructed fixed prompt template to obtain a structured input; The model building module is configured to: build a recommendation model, feed the structured input into the recommendation model, introduce a soft hint embedding of length n before the embedding layer of the recommendation model, and train the recommendation model; The model optimization module is configured to: optimize the recommendation model parameters using fine-tuning technology, repeatedly iterate to obtain a trained recommendation model; and use the trained recommendation model to make drug recommendations; The beneficial effects of the present invention are: 1. This paper first creates LLM-compatible drug term IDs through collaborative index encoding to uniquely identify each drug term. Then, a hybrid fine-tuning strategy of soft hints and LoRA is adopted to make the hint information better adapt to task requirements and efficiently fine-tune large models. Finally, the LLM is modified by adding a new output layer and an improved loss function to generate more accurate recommendation results.

[0018] 2. This invention reduces the reliance on complex frameworks in traditional drug combination recommendation methods, improving the model's usability and efficiency, making it easier for medical personnel to quickly use and understand its decision-making logic. It also achieves seamless adaptation and compatibility between drug recommendation tasks and LLM, fully leveraging the potential of LLM in the recommendation field. Furthermore, the model's prompt information can better adapt to task requirements and efficiently fine-tune large models. This method effectively improves the accuracy of drug combination recommendations while also providing a universal and efficient solution that can adapt to different scenarios and needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of the process of the personalized drug combination recommendation method using a large language model of the present invention; Figure 2 Schematic diagram of the process of the encoding method based on collaborative indexing of the present invention; Figure 3 A schematic diagram of the process of the hybrid fine-tuning strategy of the low-rank adaptation technology and soft prompting of the present invention; DETAILED DESCRIPTION

[0020] The present invention will be further described below with reference to embodiments and accompanying drawings, but is not limited thereto.

[0021] Example 1 A personalized drug combination recommendation method using a large language model, such as Figure 1 Shown, including: Step 1: Collect data from patients’ electronic health records (EHRs), including diagnosis, surgery, and medication information, and construct a dataset to facilitate model training and evaluation. Step 2: Encode the data set to obtain the encoded data; Step 3: Build fixed prompt templates. These templates serve as a bridge for the model to understand patient data and help the model capture information more accurately. The encoded data is placed into the built fixed prompt templates to obtain structured input, which helps the model process and learn data more efficiently. Step 4: Build a recommendation model, feed the structured input into the recommendation model, introduce a soft hint embedding of length n before the embedding layer of the recommendation model, and train the recommendation model; Step 5: Use fine-tuning technology to optimize the recommendation model parameters, and iterate repeatedly to obtain a trained recommendation model; use the trained recommendation model to make drug recommendations.

[0022] Example 2 The difference between the personalized drug combination recommendation method using a large language model described in Example 1 is that: Encode the data set to obtain the encoded data; Figure 2 Shown, including: The diagnostic and surgical information in the dataset is encoded using a name-based indexing strategy. Collaborative indexing relies primarily on the similarity of patient medical data, but it does not fully utilize the powerful semantic understanding capabilities of LLMs. To address this deficiency, a name index is introduced. Using a standard mapping dictionary in the medical field, the identifiers of diagnostic and surgical information (ICD, ATC4) are converted into corresponding natural language names (for example, "A02A" is converted to "antacids"). This approach is intuitive and easy to understand because it directly uses human-readable names, helping the model capture the semantic relationships between drug terms. The encoded diagnostic and surgical terms are then obtained. The drug information in the dataset was encoded using a strategy based on a combination of collaborative indexing and name indexing; Among them, the coding strategy based on collaborative indexing utilizes collaborative information (interrelated and mutually exclusive information about diagnosis, surgery, and medication) in patients' electronic health records (EHRs) and adopts spectral clustering based on spectral matrix factorization (SMF) to generate a drug term index; it includes: Construct a graph G based on the dataset, and let G = (V, E) be the co-occurrence graph, where V represents the node set of drug terms in the electronic health record, that is, the drug information, and E represents the edge set. ij ∈E represents the co-occurrence of drug term i and drug term j, and the edge weight W ij Represents the frequency of co-occurrence of i and j. The adjacency matrix A corresponding to the graph G represents the similarity between drug terms. Based on the co-occurrence frequency, the Laplace matrix L obtained by the adjacency matrix A is factorized to achieve spectral clustering, as shown below: ; Among them, L ij A represents the Laplacian matrix of drug term i and drug term j, ij Represents the adjacency matrix of drug term i and drug term j; in the spectral clustering process, solve the Laplace matrix L The eigenvalues and eigenvectors of these eigenvectors map the data points to a new feature space (usually called "spectral space"), in which the similarities between data points are more easily captured, thereby grouping the drug term nodes into different clusters, so that drug terms with more co-occurrence similarities are grouped into the same cluster; by recursively applying the spectral clustering process within the large cluster, each cluster is further subdivided into finer-grained clusters, thereby forming a hierarchical cluster structure; two key parameters are used to control the recursive clustering process: (1) N: the number of clusters generated at each level; (2) K: the maximum number of drug terms allowed in the final cluster, which serves as the stopping condition of the recursive process; when the number of drug terms contained in a cluster does not exceed K, it will not be further divided; The final result is a hierarchical tree structure, in which each non-leaf node represents a cluster created at the corresponding level, and each leaf node represents a drug term in the corresponding final cluster. The index of each drug term is obtained by concatenating the tags of the non-leaf ancestor nodes with the tags of its own leaf nodes. This indexing method means that the more frequently two terms appear, the more tags they share, thereby effectively utilizing the collaborative information in the historical sequence; Based on the strategy of combining collaborative indexing and name indexing, the two indexes are spliced together to obtain the encoded drug term ID; this indexing method not only contains the position information of the drug term in the user behavior sequence, but also incorporates its semantic information, enabling the model to understand and process the data more comprehensively; in this way, hybrid indexing enhances the model's ability to identify complex relationships between drug terms, thereby providing more accurate results in various applications.

[0023] Build fixed prompt templates. These templates serve as a bridge for the model to understand patient data and help the model capture information more accurately. The encoded data is placed into the built fixed prompt templates to obtain structured input, which helps the model process and learn data more efficiently. This includes: A prompt template T is designed to derive the language representation P (z) of the patient's electronic health record (HER), so that the LLM can understand the patient's health status. The prompt template T is filled with personalized information such as user ID and drug term ID. The construction method of the prompt template is as follows: "Patient <PATIENT ID> Total <VISIT NUM> Intensive care unit (ICU) visit records; at the first visit, the patient's diagnosis was: <DIAG CODE> … <DIAG CODE>; The treatment procedures performed are: <PROC CODE >……<PROC CODE> ; The prescribed medications are: <MED CODE> …<MED CODE> ; At the second visit... In this visit, the patient's diagnosis was: <DIAG CODE> …<DIAG CODE> ; The treatment procedures performed are: <PROC CODE> …<PROC CODE> ;Next, the patient should be prescribed the following medications:"; Use the electronic health record data and the encoded data to fill in the above template. The first segment represents the patient's history V=[V1,V2,…,V T ],in,"<PATIENT_ID> " represents the unique identifier of the patient,<VISIT_NUM> "refers to the number of hospital visits a patient has,"<DIAG_CODE> ","<PROC_CODE> "and"<MED_CODE> " represents the coded data, which are the coded diagnostic terms, surgical terms, and drug term IDs. The format of each visit is the same as the first visit. Finally, a question is asked: What medications are needed after this visit? After filling in, a structured input is obtained.

[0024] Build a recommendation model, feed the structured input into the recommendation model, introduce a soft hint embedding of length n before the embedding layer of the recommendation model, and train the recommendation model; including: Recommended models include the improved LLM model (LLM model with a new output layer and improved loss function); The improved LLM model includes: embedding layer, Transformer decoder layer, layer normalization, and linear layer; The Transformer decoder layer includes: attention mechanism, MLP (multi-layer perceptron), layer normalization; In order to increase the flexibility of prompts, a soft personalized medical prompt adjustment method is adopted to introduce learnable parameters into the input embedding space of the recommendation model. These parameters can be optimized according to conditions such as task type and dataset. The core idea is to add a learnable projection layer to the model input layer to map the original input to the semantic space represented by the prompt information. This design enables the prompt information to better adapt to task requirements. Including, the structured input obtained in step 3 is first converted into n tags {x1, x2, ..., x n}, the continuous text string is divided into discrete units for computer processing, and embedded through the improved LLM model embedding layer to capture the semantic information and contextual relationship of the text, forming a matrix X e ∈R n×e , where e represents the dimension of the embedding space; Introduce a trainable hint parameter P of length pe ∈R p×e , represents the embedding of the soft prompt, and is combined with the embedded input matrix X e Splicing to form a new input matrix [P e ;X e ]∈R (p+n)×e , where [P e ; X e ] indicates P e and X e The row-wise splicing operation uses the spliced matrix as the model input and is processed through the encoder-decoder structure. Soft hints, as virtual embedding tags, can be optimized through training to help the model better adapt to specific tasks or domains without changing the original model parameters. The input matrix [P e ;X e ] are converted into vector representations; these vectors pass through 24 Transformer decoder layers in sequence, and are layer-normalized before and after the self-attention mechanism and MLP processing in the Transformer decoder layer to ensure the stability and performance of the model; layer normalization is also performed at the end of each Transformer decoder layer to further optimize the performance of the model; finally, the output of all Transformer decoder layers passes through a linear layer to generate the final classification score.

[0025] Use a linear layer with sigmoid to output the probability of each drug as follows: ; in, , is the predicted medication probability and hidden state of the last Transformer decoder layer in the improved LLM model, R represents a real number matrix, |M| represents the total number of all drugs, represents the dimension of the hidden state, is a learnable weight matrix, Represents the sigmoid function. For the final recommendation, set a threshold γ. When y k >γ, drug k will be included in the prescription drug set, where y k Represents the predicted medication probability of the Kth drug.

[0026] During the training of the recommendation model, a supervised fine-tuning (SFT) strategy is used, which uses labeled data to adjust the improved LLM model to make it more suitable for the specific task of drug combination recommendation. At the same time, in order to make the model more accurately adapt to the output layer of the modified LLM, the loss function is calculated by comparing the difference between the predicted output of the recommendation model and the actual target output. The loss function is set to: ; Where N is the number of drug categories, y (i) is the one-hot encoding of the true label, is the one-hot encoding of the recommendation model prediction.

[0027] Use fine-tuning technology to optimize the recommendation model parameters, iterate repeatedly to obtain a trained recommendation model; use the trained recommendation model to make drug recommendations; including: To further reduce the cost of fine-tuning after introducing the soft personalized medical prompt adjustment method, the Low-Rank Adaptation of Large Language Models (LoRA) technology is used in the recommendation model training process. Low-Rank Adaptation technology updates the recommendation model weights by decomposing them into low-rank components. In this way, only the weights of the task-related subspace are adjusted, as shown below: ; Among them, W new is the updated weight, W old is the original weight, ∆W is the low-rank update matrix; in this way, the update range of the model parameters is limited, thereby reducing the consumption of computing resources; the parameter update matrix ∆W of each layer is decomposed into two low-rank matrices: ; Where A ∈ R r×k and B ∈ R d×r are two low-rank matrices; generally speaking, r ≪ min(d , k), d represents the dimension of the input feature, k represents the dimension of the output feature, and r represents the rank of the low-rank matrix; let Represents a set of trainable matrices, where Ai and Bi are the low-rank matrices A and B of the i-th layer, respectively, and L is the number of layers of the low-rank matrix. represents the parameter set of all low-rank matrices; During the training process, the model first performs forward propagation to calculate the loss between the predicted output and the true label; then, the backpropagation algorithm calculates the partial derivative of the loss function with respect to each parameter through the chain rule. These partial derivatives indicate the contribution of the parameter to the loss; finally, using this gradient information, the model parameters are updated through the optimization algorithm (gradient descent) to reduce the loss and improve the model performance; the training is iterated continuously until the model converges to a satisfactory performance level, resulting in a trained recommendation model.

[0028] During the testing phase, the patient's previous visit data, along with their current diagnosis and surgery information, are encoded in the test set. This encoded data is then populated into a pre-designed template, along with additional input data of length n. This prepared input data is then fed into the trained model, which performs deep learning and analysis based on this information. Ultimately, the model generates the medication combination appropriate for the current visit. This result will provide powerful support for clinicians, helping them make more precise treatment decisions.

[0029] In this paper, we first make key adjustments to the original model at the initial stage of model training. We replace its language generation head with a classification head and introduce a soft hint embedding of length n before the embedding layer. This step is to better adapt to the specific task of drug recommendation. Next, all the historical visit records of each patient in the dataset, including diagnosis (D1 to D t ), surgery (P1 to P t ) and recommended drugs (M1 to M t ), encoded using a collaborative indexing-based encoding strategy. The encoded data is filled into a pre-designed prompt template T to construct a structured input format. During the training process, for each batch of samples, the present invention not only retains the original data length, but also additionally fills the input data with a length of n, and then inputs these data into the LLM to obtain the hidden state h of each sample. Subsequently, through the optimized classification head, the model is able to output the probability of each drug being recommended. It is worth noting that during the training process, the present invention freezes all pre-trained parameters of the LLM to maintain the stability of the model. Only the parameters of the classification head and soft prompt embedding are backpropagated and fine-tuned, and this process will be repeated for all batches throughout the training cycle. After this training process, a fine-tuned model is obtained.

[0030] During the inference phase, the patient's previous visit data [V1, V2, ...Vt-1], the current diagnosis Dt, and the procedure Pt are again encoded using a collaborative indexing-based encoding strategy. This encoded data is then populated into our pre-designed prompt template T. Similar to the training phase, input data of length n is padded and fed into the trained model to generate the medication recommendation Mt for the current visit. This process ensures that the model can provide personalized medication recommendations based on the latest patient data.

[0031] In our experiments, we used two datasets: MIMIC-III and MIMIC-IV. Both MIMIC-III and MIMIC-IV are derived from the Medical Information Intensive Care Repository (MIMIC). Specifically, MIMIC-III compiles hospitalization records for 46,520 patients between 2001 and 2012, while MIMIC-IV contains data on 299,712 patients collected from 2008 to 2019. We compared our model with baseline methods such as logistic regression (LR), ensemble classifier chains (ECC), LEAP, GameNet, SafeDrug, 4SDrug, MICRON, and MoleRec.

[0032] Logistic Regression (LR): LR is an instance-based classifier with L2 regularization.

[0033] Ensemble Classifier Chain (ECC): ECC is a multi-label model that arranges LR classifiers into a chain. Each classifier uses the prediction of the previous classifier in the chain as a feature.

[0034] RETAIN: RETAIN designs a two-level attention model to improve the accuracy and interpretability of clinical variable predictions. We achieve this by adding representations of diagnosis and procedure for each visit.

[0035] LEAP: LEAP uses an LSTM-based generative model to make drug recommendations based on diagnostic information.

[0036] GAMENet: GAMENet adopts a memory bank to integrate global drug interaction and drug-drug interaction knowledge. In our implementation, we replace the retrieval representation from patient history with the representation from patient similarity for those patients in a single visit.

[0037] SafeDrug: SafeDrug encodes drugs using their molecular structure and adds direct drug-drug interaction controls during training.

[0038] 4SDrug: 4SDrug aims to recommend small drug combinations to ensure fewer drug-drug interactions.

[0039] MoleRec: MoleRec models the interactions between molecular substructures and the dependencies between a patient's health status and these substructures.

[0040] We used three popular metrics for drug recommendation: Jaccard similarity score, F1 score, and precision-recall area under the curve (PRAUC). Table 1 shows the performance comparison on the MIMIC-III dataset, and Table 2 shows the performance comparison on the MIMIC-IV dataset, as shown below: Table 1 Table 2 Example 3 A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the personalized drug combination recommendation method using a large language model described in Example 1 or 2.

[0041] Example 4 A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the personalized drug combination recommendation method using a large language model described in Example 1 or 2 are implemented.

[0042] Example 4 A personalized drug combination recommendation system using a large language model, including: The data collection module is configured to: collect data from electronic health records, including diagnosis information, surgery information, and medication information, and construct the collected data into a dataset; The encoding module is configured to: encode the data set to obtain encoded data; The prompt template module is configured to: construct a fixed prompt template and put the encoded data into the constructed fixed prompt template to obtain a structured input; The model building module is configured to: build a recommendation model, feed the structured input into the recommendation model, introduce a soft hint embedding of length n before the embedding layer of the recommendation model, and train the recommendation model; The model optimization module is configured to: optimize the recommendation model parameters using fine-tuning technology, repeatedly iterate to obtain a trained recommendation model; and use the trained recommendation model to make drug recommendations.

Claims

1. A personalized drug combination recommendation method using a large language model, characterized in that: include: Step 1: Collect data from electronic health records, including diagnosis information, surgery information, and medication information, and construct the collected data into a dataset; Step 2: Encode the data set to obtain the encoded data; Step 3: Build a fixed prompt template and put the encoded data into the built fixed prompt template to obtain structured input; Step 4: Build a recommendation model, feed the structured input into the recommendation model, introduce a soft hint embedding of length n before the embedding layer of the recommendation model, and train the recommendation model; Step 5: Use fine-tuning technology to optimize the recommendation model parameters, and iterate repeatedly to obtain a trained recommendation model; use the trained recommendation model to make drug recommendations.

2. The personalized drug combination recommendation method using a large language model according to claim 1, characterized in that: Encode the data set to obtain encoded data; including: The diagnostic information and surgical information in the dataset are encoded using a name indexing strategy, that is, the identifiers of the diagnostic information and surgical information are converted into corresponding natural language names to obtain the encoded diagnostic terms and surgical terms; The drug information in the dataset was encoded using a strategy based on a combination of collaborative indexing and name indexing; Among them, the coding strategy based on collaborative indexing utilizes collaborative information in electronic health records and adopts spectral clustering based on spectral matrix decomposition to generate a drug term index; Based on the strategy of combining collaborative index and name index, the two indexes are spliced together to obtain the encoded drug term ID.

3. The personalized drug combination recommendation method using a large language model according to claim 2, characterized in that: Build a recommendation model, feed the structured input into the recommendation model, introduce a soft hint embedding of length n before the embedding layer of the recommendation model, and train the recommendation model; including: Recommended models include the improved LLM model; The improved LLM model includes: embedding layer, Transformer decoder layer, layer normalization, and linear layer; The Transformer decoder layer includes: attention mechanism, MLP, layer normalization; A soft personalized medical prompt adjustment method is adopted to introduce learnable parameters into the input embedding space of the recommendation model, including: the structured input obtained in step 3 is first converted into n tags {x1, x2, ..., x n }, the continuous text string is divided into discrete units for computer processing, and embedded through the improved LLM model embedding layer to form a matrix X e ∈R n×e , where e represents the dimension of the embedding space; Introduce a trainable hint parameter P of length p e ∈R p×e , represents the embedding of the soft prompt, and is combined with the embedded input matrix X e Splicing to form a new input matrix [P e ;X e ]∈R (p+n)×e , where [P e ; X e ] indicates P e and X e The operation of splicing in the row direction; The input matrix [P e ;X e ] is converted into a vector representation; these vectors pass through 24 Transformer decoder layers in sequence, and are layer-normalized before and after the self-attention mechanism and MLP processing in the Transformer decoder layer; layer normalization is also performed at the end of each Transformer decoder layer; finally, the output of all Transformer decoder layers passes through a linear layer to generate the final classification score.

4. The personalized drug combination recommendation method using a large language model according to claim 3, characterized in that: Use a linear layer with sigmoid to output the probability of each drug, as shown below: ； in, , is the predicted medication probability and hidden state of the last Transformer decoder layer in the improved LLM model, R represents a real number matrix, |M| represents the total number of all drugs, represents the dimension of the hidden state, is a learnable weight matrix, Represents the sigmoid function. For the final recommendation, set a threshold γ. When y k >γ, drug k will be included in the prescription drug set, where y k Represents the predicted medication probability of the Kth drug.

5. The personalized drug combination recommendation method using a large language model according to claim 4, characterized in that: During the training of the recommendation model, a supervised fine-tuning strategy is used, that is, labeled data is used to adjust the improved LLM model. At the same time, the loss function is calculated by comparing the difference between the predicted output of the recommendation model and the actual target output. The loss function is set to: ； Where N is the number of drug categories, y (i) is the one-hot encoding of the true label, is the one-hot encoding of the recommendation model prediction.

6. The personalized drug combination recommendation method using a large language model according to claim 5, characterized in that: Use fine-tuning technology to optimize the recommendation model parameters, iterate repeatedly to obtain a trained recommendation model; use the trained recommendation model to make drug recommendations; including: Low-rank adaptation technology is used in the recommendation model training process. Low-rank adaptation technology updates the model weights by decomposing them into low-rank components, as shown below: ； Among them, W new is the updated weight, W old is the original weight, ∆W is the low-rank update matrix; the parameter update matrix ∆W of each layer is decomposed into two low-rank matrices: ； Where A ∈ R r × k and B ∈ R d × r are two low-rank matrices; r ≪ min(d , k), d represents the dimension of the input feature, k represents the dimension of the output feature, and r represents the rank of the low-rank matrix; let Represents a set of trainable matrices, where Ai and Bi are the low-rank matrices A and B of the i-th layer, respectively, and L is the number of layers of the low-rank matrix. Represents the set of parameters of all low-rank matrices.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the personalized drug combination recommendation method using a large language model according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for recommending personalized drug combinations using a large language model according to any one of claims 1 to 6 are implemented.

9. A personalized drug combination recommendation system using a large language model, characterized in that: include: The data collection module is configured to: collect data from electronic health records, including diagnosis information, surgery information, and medication information, and construct the collected data into a dataset; The encoding module is configured to: encode the data set to obtain encoded data; The prompt template module is configured to: construct a fixed prompt template and put the encoded data into the constructed fixed prompt template to obtain a structured input; The model building module is configured to: build a recommendation model, feed the structured input into the recommendation model, introduce a soft hint embedding of length n before the embedding layer of the recommendation model, and train the recommendation model; The model optimization module is configured to: optimize the recommendation model parameters using fine-tuning technology, repeatedly iterate to obtain a trained recommendation model; and use the trained recommendation model to make drug recommendations.

Images