Knowledge-enhanced drug recommendation method based on priori knowledge and security perception constraint

Abstract

The invention discloses a knowledge-enhanced drug recommendation method based on priori knowledge and security perception constraints. The method comprises the following steps: firstly, carrying out data preprocessing, converting drug codes from NDC to ATC-3 level standards, carrying out standardization processing on disease and program codes by adopting an ICD-9 code system, and dividing a data set; in the embedding layer, a special input sequence is designed, segmentation embedding and correlation embedding are combined, medical priori knowledge is utilized to construct a correlation weight matrix, and complex correlation between diseases and program codes is effectively captured. Through two self-supervised pre-training tasks of ICD sequence prediction and mask sequence reconstruction, a clinical code relationship is mined, and the understanding of the model on clinical codes is enhanced. In the fine tuning stage, constraint learning is fused, a multi-label marginal loss function optimization model is used, the drug taboo relation is fused into a loss function through annotation consistency constraint and taboo constraint, recommendation accuracy and safety are balanced, finally, drugs with the probability larger than 0.5 are predicted and recommended through an MLP layer, and powerful support is provided for precise medical treatment.

Description

Technical Field

[0001] The present invention relates to technical fields such as drug recommendation, machine learning, deep learning, knowledge graphs, and in particular to a knowledge-enhanced drug recommendation method based on the collaborative optimization of prior knowledge fusion and safety perception constraints; in particular, to a knowledge-enhanced Transformer drug recommendation method based on the collaborative optimization of prior knowledge fusion and safety perception constraints. Background Art

[0002] In the modern healthcare system, precision medicine has become a key development direction in the medical field, and the importance of drug recommendation models, as a key component of precision medicine, is self-evident. By comprehensively analyzing multiple sources of data, including a patient's disease characteristics, medical history, genetic information, and lifestyle habits, drug recommendation models can provide doctors with scientifically sound drug treatment recommendations. This not only helps doctors quickly screen the most suitable medication for a patient, avoiding the time and financial costs of trial-and-error treatment, but also plays a key role in improving treatment outcomes, reducing adverse drug reactions, and optimizing medical resource allocation. Accurate drug recommendations are particularly important when faced with complex conditions and multiple drug options, effectively improving patients' treatment experience and prognosis.

[0003] The drug-drug interaction (DDI) rate is a key indicator for measuring the likelihood of drug combinations interacting in clinical applications. It reflects the probability of adverse events caused by physical, chemical, or pharmacological interactions between two or more drugs when used simultaneously. In an ideal drug recommendation scenario, a drug recommendation system should be able to accurately predict and control DDIs, ensuring that recommended drug combinations are highly safe and effective, and maximizing patient medication safety.

[0004] However, most current drug recommendation systems rely primarily on data-driven approaches, which have exposed numerous problems in practical applications. For one thing, the quality of medical data varies greatly. Clinical data comes from a wide range of sources, and data standards and recording methods vary significantly across different medical institutions. For example, drug names may be recorded in multiple different ways, with some using generic names, others using trade names, and even typographical errors. This can easily lead to confusion during data processing, severely impacting the accuracy of recommendations. Furthermore, data may contain missing values ​​and noise, further reducing data usability. Furthermore, data-driven approaches have significant limitations when dealing with complex relationships between drugs. Drug-drug interactions are complex processes involving multiple mechanisms, including pharmacology and pharmacokinetics. Purely data-driven models often make recommendations based only on superficial correlations in the data, failing to deeply understand the underlying mechanisms of drug interactions. For example, in a study on a combination of a certain type of commonly used antihypertensive drug and lipid-lowering drug, because the data did not exclude factors such as patients taking other over-the-counter drugs or health products with potential interactions at the same time, the data-driven drug recommendation system had an error of up to 40% in assessing the DDI rate of this combination, which undoubtedly poses a potential threat to patients' health.

[0005] While some research has recognized these issues and attempted to improve drug recommendation models by incorporating prior knowledge and constraints, these models still suffer from numerous flaws. Regarding knowledge representation, existing methods often simply incorporate rules or database information into the model, lacking a deep understanding and flexible application of knowledge, making it difficult to accurately represent the complexity and diversity of medical knowledge. When it comes to integration with recommendation algorithms, these approaches are often ineffective and prone to introducing noise, resulting in limited improvements in model performance. Furthermore, with the continuous advancement of medical research and the emergence of new drugs, these models have become less adaptable to new knowledge, unable to update knowledge in a timely manner, and unable to meet actual clinical needs.

[0006] To address these issues, the present invention proposes a knowledge-enhanced drug recommendation model (KETMed) based on the collaborative optimization of prior knowledge fusion and safety-aware constraints. This model innovatively employs a pre-training-fine-tuning learning paradigm, fully leveraging medical prior knowledge and constraints to improve the accuracy and safety of drug recommendations. At the embedding layer, the model initializes a correlation weight matrix with prior knowledge, successfully capturing the complex relationships between disease and program code. This model leverages medical expertise and addresses the inadequate use of prior knowledge in traditional models. At the prediction layer, drug-contraindication relationships are formalized as constraints in the loss function, allowing the drug recommendation process to precisely balance the therapeutic benefits of a drug with the risk of contraindications in the presence of comorbidities, significantly improving the safety and accuracy of recommendations. Testing on two public, real-world datasets, MIMIC-III and MIMIC-IV, demonstrates that KETMed reduces safety risk by 2.04% and improves accuracy by 5.2% compared to the next-best baseline model. This represents a new breakthrough in the field of drug recommendation and is expected to further advance precision medicine. Summary of the Invention

[0007] This invention focuses on solving the limitations of traditional data-driven drug recommendation models and early knowledge enhancement models. It aims to significantly improve the accuracy and safety of drug recommendations by integrating medical prior knowledge with constrained learning, thereby providing more reliable support for clinical drug recommendations.

[0008] The technical solution of this invention is to propose a knowledge-enhanced drug recommendation model (KETMed) based on the collaborative optimization of prior knowledge fusion and safety-aware constraints. This model effectively integrates medical prior knowledge and constraint learning mechanisms through multi-step and multi-module collaboration to achieve accurate and safe drug recommendations. The specific steps are as follows:

[0009] (1) Data preprocessing: Carefully screen and obtain high-reliability real electronic health record (EHR) datasets from MIMIC-III and MIMIC-IV. These datasets cover a wealth of patient medical information and provide a solid data foundation for model training. In response to the diversity of drug codes and the inconsistency of disease and procedure codes, drug codes are converted from the NDC specification to the ATC-3 level standard. At the same time, the ICD-9 code system is used to standardize disease and procedure codes to unify the data format and eliminate the interference caused by data differences. On this basis, the data is randomly divided into training set, validation set, and test set according to the scientific ratio of 4:1:1 to ensure the independence and representativeness of each dataset, providing effective guarantees for model training, optimization, and evaluation.

[0010] (2) Model embedding layer construction:

[0011] a. Design input sequence: Design targeted input sequences based on the different situations of the patient's medical records. Construct a single medical record input sequence:

[0012]

[0013] [CLS] is used as the starting special marker to effectively integrate the overall information; [SEP] is used to clearly separate the disease code d and the procedure code p to ensure accurate distinction of information. Construct the input sequence of multiple medical records:

[0014]

[0015] The special tag [END_VISIT] is specially introduced, and combined with special position coding, it can accurately distinguish the sequence and correlation between different medical records, helping the model to better capture changing trends.

[0016] b. Introducing additional embeddings: To enhance the model's ability to understand different types of medical information, segmented embedding technology is used to assign unique embedding vectors to diseases and procedures, enabling the model to accurately identify and process the context of different categories of information. At the same time, we deeply explore medical prior knowledge and extract information related to diseases and procedures from resources such as authoritative medical guidelines, large-scale clinical research results, and professional medical literature to construct a domain knowledge graph. We assign initial weights to the associated edges in the graph by calculating Jaccard similarity, and then modify the matrix by combining semantic similarity calculations. The formula is:

[0017] A i,j =α·ClinicalWeight(d i ,p j )+(1-α)·SemanticSim(d i ,p j ) (3)

[0018] Where α is the balance coefficient, and its value range is [0,1], so as to achieve more accurate correlation embedding and effectively capture the complex relationship between disease and program code.

[0019] c. Generate Representation: Leveraging the powerful feature extraction capabilities of the Transformer encoder, the embedded input sequence is deeply processed. The outputs of the token, segmentation, and relevance embedding layers are fused and added, and then further transformed by the Transformer encoding layer to generate a comprehensive and accurate representation of the clinical condition, providing high-quality feature data for subsequent drug recommendation tasks.

[0020] (3) Model pre-training:

[0021] a. ICD Sequence Prediction (ISP): By designing whether the disease and program ICD code sequences belong to the same task, a negative sample sequence is constructed to simulate data changes in real scenarios. Binary Cross Entropy Loss (BCE) is used as the loss function, and the formula is:

[0022]

[0023] During the training process, the model continuously learns and optimizes, gradually understanding the contextual dependencies between clinical codes, thereby improving its ability to capture potential connections between clinical conditions.

[0024] b. Masked Sequence Reconstruction (MSR): Randomly select some ICD codes in the input sequence for masking, and let the model reconstruct the masked codes based on the context information. The cross entropy loss is used to calculate the difference between the predicted and true code distributions. The formula is:

[0025]

[0026] In this way, the model can deeply learn the semantic information and contextual relationships of clinical codes, and enhance the understanding and memory ability of clinical codes.

[0027] (4) Model fine-tuning:

[0028] a. Multi-label marginal loss optimization: Using multi-label marginal loss function:

[0029]

[0030] Fine-tune the pre-trained model. This function effectively ensures that correct drug combinations are scored significantly higher than incorrect combinations. By continuously optimizing model parameters, the model becomes more accurate in predicting drug combinations and improves the accuracy of recommendations.

[0031] b. Constrained learning: The drug-contraindication relationship is cleverly integrated into the loss function in the form of annotation consistency constraints and contraindication constraints. The annotation consistency constraint loss function is:

[0032]

[0033] By strengthening the relationship between disease codes and drug codes, the model's understanding of disease-drug associations is improved. The taboo constraint loss function is:

[0034]

[0035] When there are certain comorbidities, it is strictly forbidden to recommend contraindicated drugs. The total loss function is:

[0036] L constr =LA +λ*L B (9)

[0037] Here, λ is a non-negative coefficient that controls each loss term. By dynamically adjusting the λ value, precise control of the model learning process is achieved, so that the model can fully consider the risk of drug contraindications during the recommendation process.

[0038] (5) Model prediction:

[0039] a. Balancing accuracy and security: Combining multi-label marginal loss functions L by weighted sum multi And the total loss function L constr , the formula is:

[0040] L=(1-β)L multi +β*L constr (10)

[0041] β is a hyperparameter that balances the contributions of different losses. By properly adjusting β, we can maximize the safety of recommendations while ensuring the accuracy of the recommendations, ensuring that the recommended drugs meet the needs while avoiding potential drug interaction risks.

[0042] b. Drug Recommendation: A multi-layer perceptron (MLP) layer is integrated, using the representation generated by the fine-tuned model as input to accurately predict the recommendation probability of each drug. The recommendation threshold is set to 0.5, and drugs with predicted probabilities greater than this threshold are included in the final recommended drug set. The output is represented as:

[0043]

[0044] This provides doctors with a concise and effective list of drug recommendations to assist in clinical decision-making.

[0045] Compared with the prior art, the present invention has the following significant advantages:

[0046] (1) Innovation in prior knowledge integration: In the embedding layer, the correlation weight matrix is ​​initialized with the help of medical prior knowledge to effectively mine and capture the complex and subtle associations between diseases and program codes, making up for the shortcomings of traditional models in utilizing prior knowledge and making the model's recommendations more scientific and reasonable.

[0047] (2) Constrained learning improves safety: An innovative constrained learning mechanism is introduced to transform the drug-contraindication relationship into a constraint in the loss function, accurately balancing the drug treatment effect and the risk of comorbidity contraindications during the drug recommendation process, significantly improving the safety of recommendations and reducing medication risks.

[0048] (3) Advantages of the Pre-training-Fine-tuning Framework: This framework utilizes an advanced pre-training-fine-tuning framework, employing two self-supervised pre-training tasks: ICD sequence prediction and mask sequence reconstruction. This enhances the model’s understanding of clinical codes and its feature extraction capabilities, providing more robust and representative feature representations for drug recommendations. Experiments demonstrate that this framework performs exceptionally well on real-world datasets, reducing safety risks by 2.04% and increasing accuracy by 5.2% compared to existing baseline models.

[0049] (4) Ablation experiments to verify key components: Systematic ablation experiments were conducted on each key component of the model, which clearly verified the key role of pre-training tasks, special position encoding design, and embedding of medical prior knowledge in improving model performance, providing a strong theoretical basis and practical guidance for further optimization and improvement of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention will be better understood from the following detailed description of the embodiments of the present invention in conjunction with the accompanying drawings.

[0051] Figure 1 It is a flow chart of the overall steps of the present invention.

[0052] Figure 2 It is the overall framework diagram of the present invention.

[0053] Figure 3 It is the hyperparameter learning graph of the present invention. DETAILED DESCRIPTION

[0054] The implementation of this invention involves several key steps, each carefully designed and debugged to ensure the efficient and accurate operation of the knowledge-enhanced drug recommendation framework KETMed, which is based on the collaborative optimization of prior knowledge fusion and safety-aware constraints. The details are as follows:

[0055] (1) Data preprocessing:

[0056] a. Data Acquisition: Data was obtained from the publicly available medical databases MIMIC-III (containing multidimensional clinical data on approximately 61,000 intensive care unit (ICU) patients between 2001 and 2012) and MIMIC-IV (which optimized the data collection process and expanded the data scope, adding approximately 46,000 ICU patient records between 2013 and 2019). This data covers patient diagnosis information, medication records, and test results. It is stored in formats such as CSV or JSON and contains key fields such as "patient_id," "diagnosis_code," "medication_code," and "procedure_code."

[0057] b. Code conversion and standardization: Data was read and processed using the pandas library in Python. For medication codes, a mapping function was developed to convert NDC codes in the "medication_code" field to ATC-3 standard codes based on the NDC-ATC-3 standard correspondence table. For disease and procedure codes, the "diagnosis_code" and "procedure_code" fields were standardized according to the ICD-9 code system, removing non-compliant characters and incorrectly formatted data.

[0058] c. Dataset division: Use the train_test_split function in the sklearn library to randomly divide the processed data into training set, validation set, and test set in a ratio of 4:1:1.

[0059] (2) Embedding layer construction

[0060] a. Design the input sequence: In Python, define the build_single_visit_sequence and build_multi_visit_sequence functions to construct the input sequence. For patients with a single visit, use the build_single_visit_sequence function, which takes the disease code and procedure code as input. Within the function, the disease code and procedure code are concatenated using the cls_token (start token [CLS]) and sep_token (separation token [SEP]) to generate a single-visit input sequence in a specific format. For patients with multiple visits, the build_multi_visit_sequence function takes disease_codes (a list of multiple disease codes) and procedure_codes (a list of multiple procedure codes) as parameters. Within the loop, each iteration uses sep_token to separate the disease and procedure codes, and uses end_visit_token (end visit marker [END_VISIT]) to distinguish between different visit records, ultimately constructing a qualified multiple visit input sequence. Furthermore, a custom positional encoding function is used to assign an increasing positional encoding to each code in the visit record based on the order of the visits, helping the model better understand the sequential relationship between different visit records.

[0061] b. Introducing additional embeddings: Segmented embedding uses the PyTorch framework to create two embedding matrices, disease_embedding and procedure_embedding. disease_embedding is used to map disease codes to a specific low-dimensional vector space, and procedure_embedding performs the same operation on program codes. In this way, segmented embedding is achieved, enabling the model to distinguish between input diseases and program codes. Correlation embedding collects disease and procedure association information from authoritative medical databases, medical literature and other resources to construct a correlation matrix. When calculating the correlation weight matrix, first convert the obtained disease_descriptions (disease text description list) and procedure_descriptions (procedure text description list) into vector form using TfidfVectorizer to obtain disease_vectors and procedure_vectors. Then, the semantic similarity matrix SemanticSim is calculated through cosine_similarity. Construct the clinical weight matrix ClinicalWeight, set the balance coefficient α = 0.75, and follow formula A i,j =α·ClinicalWeight(d i ,p j )+(1-α)·SemanticSim(d i ,p j ) is corrected and the final correlation weight matrix is ​​obtained.

[0062] c. Generate patient representations: Build the PatientRepresentationModel class based on PyTorch to generate patient representations. In the class's initialization function, integrate disease_embedding, procedure_embedding, and relevance_embedding into embedding_layers and define a TransformerEncoder. In the forward function, pass the input sequence through each embedding layer to generate embedded_sequences, then sum them to generate combined_embedding. This is then processed by the transformer_encoder to output a patient representation that comprehensively reflects the patient's clinical condition. When initializing the model, set parameters such as num_heads = 4, num_layers = 4, and embed_dim = 512.

[0063] (3) Model pre-training

[0064] a. ICD Sequence Prediction (ISP): When constructing the negative sample sequence, one or more ICD codes in the disease or procedure sequence are randomly selected and replaced from different patient samples. During the training process, BCELoss of PyTorch is used as the loss function (criterion), and the Adam optimizer is used to train the model with a learning rate of 1e-5. In the training loop, at each iteration, first use optimizer.zero_grad() to clear the gradients, obtain the output by predicting the input_sequence through the model, calculate the loss between the predicted result output and the true label label, then perform backpropagation to calculate the gradients through loss.backward(), and finally update the model parameters using optimizer.step() to enable the model to learn the context dependencies of clinical codes.

[0065] b. Masked Sequence Reconstruction (MSR): Randomly mask some ICD codes by defining the mask_icd_codes function. Inside the function, with input_sequence and mask_prob (mask probability) as parameters, iterate through each code in the sequence, and determine whether to perform the masking operation according to the condition torch.rand(1).item() < mask_prob. If the condition is met, replace the code with the mask token corresponding to mask_token_id to generate the masked_sequence. In the training loop, the cross-entropy loss is used to calculate the difference between the code distribution predicted by the model and the true code distribution, and continuously optimize the model's learning ability for clinical code semantic information and context relationships.

[0066] (4) Model Fine-tuning and Prediction

[0067] a. Multi-label margin loss optimization: Define the multi_label_margin_loss function in PyTorch to implement multi-label margin loss optimization. The function takes predicted_logits (the logarithmic probability predicted by the model), true_labels (the actual label), and margin (margin value, defaulting to 1.0) as parameters. It iterates over all categories, compares the score difference between the correct and incorrect drug combinations, calculates and accumulates the loss using the F.relu function, and finally returns the average loss value. In the training loop, the number of iterations is fine_tune_epochs. At each iteration, the gradient is cleared using optimizer.zero_grad(). The model predicts the input_sequence to obtain the output. The loss between the output and the true_labels is calculated. Backpropagation is performed using loss.backward() and the optimizer.step() function is used to update the parameters to optimize model performance.

[0068] b. Constraint learning: Incorporate the drug-contraindication relationship into the loss function in the form of annotation consistency constraints and taboo constraints. Define the annotation_consistency_loss function to calculate the annotation consistency constraint loss (LA). The function receives model_output (model output), disease_codes (disease code), drug_codes (drug code) and phi (balanced loss weight) as parameters. In the loop, the relationship between the disease and the drug is obtained through the calculate_relation function (assuming that this function calculates the disease-drug relationship). The loss value is calculated and accumulated according to the formula. The taboo constraint loss function (LB) is defined and calculated according to the specific drug contraindication logic. The total loss function is L constr =L A +λ*L B ,During the training process, parameters such as λ are continuously adjusted to balance the loss terms and dynamically optimize the model.

[0069] c. Balancing accuracy and security: Combining two objective functions by weighted sum L = (1-β)L multi +β*L constr To balance recommendation accuracy and safety, β is initially set to 0.5. During training, the β value is dynamically adjusted based on the performance indicators of the validation set to achieve the optimal balance, ensuring that the model ensures both accuracy and safety when recommending drugs.

[0070] d. Drug recommendation: Build an MLP class in PyTorch to perform drug recommendations. The MLP class contains two fully connected layers, fc1 and fc2, with a ReLU activation function in between. When initializing the MLP layer, set input_dim = 512 (the input dimension is consistent with the patient representation dimension), hidden_dim = 256 (the hidden layer dimension), and output_dim = num_drugs (the output dimension is the number of drug types). During the drug recommendation process, first obtain the patient_representation (patient representation) generated by the model, input it into the MLP layer to obtain drug_predictions (drug prediction probability), and finally filter out drugs with a probability greater than the threshold of 0.5 to form the final recommended drug set FINAL.

[0071] The present invention conducts experiments in the RTX3060 GPU environment and uses the PyTorch framework to implement model construction, training and testing. All model hyperparameters are determined based on the test set and validation set indicators. KETMed is pre-trained on the training set in sequence, first performing the ISP task and then the MSR task, with 40 rounds of pre-training for each. The Transformer encoder in the model consists of 4 layers and 4 attention heads, and embed_dim is set to 512. The initial value of the balance coefficient α is set to 0.75, and the weight of the loss function is 0. λ and β are initialized to 0.05, 0.6, and 0.5, respectively, the learning rate is set to 1e-5, and the weight decay is 0.1.

[0072] The embodiments of the present invention are described in detail above. The present invention is explained in this article using specific implementation methods. The description of the above embodiments is only used to help understand the method of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A knowledge-enhanced drug recommendation method based on prior knowledge and safety perception constraints, characterized by: The implementation steps of this method are as follows: (1) Data processing: Structured medical data were obtained from the MIMIC-III and MIMIC-IV databases, and data cleaning and standardization were performed to address inconsistencies in data records and format inconsistencies to ensure the high quality, accuracy, and completeness of the analyzed data. (2) Embedding layer design steps: Based on the processed data, a special input sequence is constructed, and segmented embedding and correlation embedding methods are introduced to effectively express the characteristics of medical data. The data is further processed through the Transformer encoder to obtain a more complete and accurate data feature representation; (3) Model pre-training step: Based on the feature representation obtained by the embedding layer, a self-supervised pre-training task is designed to obtain robust feature representation by mining the contextual dependencies and latent semantic associations between data features, thereby effectively improving the performance of subsequent recommendation models; (4) Model fine-tuning step: Using the pre-trained model as a basis, the model parameters are optimized by integrating the constrained learning strategy to achieve the best balance between recommendation accuracy and recommendation security; (5) Recommendation result prediction step: After fine-tuning the model, the multi-layer perceptron (MLP) is used to predict the probability of drug recommendation, and a set threshold is used to determine the drugs to be included in the recommended drug list, thereby completing accurate and safe drug recommendations.

2. The knowledge-enhanced drug recommendation method based on prior knowledge and safety perception constraints according to claim 1 is characterized in that: In step (1), data preprocessing includes: (1) Data collection and organization: First, structured multidimensional medical record information was obtained from the MIMIC-III and MIMIC-IV databases. In-depth data cleaning and standardization were performed to address the issues of inconsistent drug name records and data format differences. By removing duplicate, erroneous, and incomplete records, the high quality and uniformity of the data were ensured, providing a solid data foundation for subsequent data analysis and modeling. (2) Code conversion and standardization: After the data is collated, the original drug codes are converted from the National Drug Code (NDC) to the International Anatomical Therapeutic Chemical Classification (ATC)-3 standard code system to unify the analysis standards. The International Classification of Diseases, Ninth Edition (ICD-9) code system is used to standardize the coding information of each medical record to ensure that data from different sources are highly consistent and identifiable in the subsequent model application process. (3) Dataset division and management: To evaluate and optimize the performance of the recommendation model, the dataset is randomly divided into training set, validation set, and test set in a ratio of 4:1:1 based on the results of data processing and standardization. The training set is used for model parameter learning and feature recognition, the validation set is used for parameter tuning and model optimization to avoid overfitting, and the test set is used to evaluate the final model performance to verify the performance stability and generalization ability of the model on different data sets.

3. The knowledge-enhanced drug recommendation method based on prior knowledge and safety perception constraints according to claim 1 is characterized in that: In step (2), the embedding layer construction includes: (1) Input sequence design: Based on data preprocessing, the input sequence is constructed for the single visit record data. Among them, [CLS] is a special marker placed at the beginning of the medical code sequence, which represents the overall characteristics of the data; [SEP] is a separator between the feature 1 and feature 2 coding sequences, which plays a role in clearly distinguishing different feature codes; Indicates sequence splicing operation; d represents disease feature 1, p represents program feature 2; for multiple visit data Introducing the [END_VISIT] special marker to separate different medical records, and designing a special position encoding method adapted to it to accurately represent the order and relationship between different medical records; (2) Segment embedding: After the sequence is constructed, segment embedding technology is used to effectively distinguish between input features 1 and 2. By assigning unique embedding vectors to features 1 and 2, the model can accurately identify the context of different types of medical information, and thus more accurately capture the interactive relationship between different category codes, improving the model's ability to understand complex medical information, so as to facilitate the efficient processing of the subsequent Transformer encoder; (3) Correlation embedding: After completing the segmented embedding of the input sequence, the correlation information of feature 1 and feature 2 is extracted from authoritative medical resources to construct a domain knowledge graph; structured data is extracted from the knowledge graph to form an initial set of two-feature association pairs; each association edge in the set is assigned a weight by calculating the Jaccard similarity, and high weights are assigned to high-frequency co-occurrence and clear treatment guideline-supported two-feature association pairs, while low-frequency or atypical associations are assigned lower weights; at the same time, the cosine similarity of the description texts of feature 1 and feature 2 is calculated using the medical term vector, and the initial weight matrix is ​​modified. The formula is A i,j =α·ClinicalWeight(d i ,p j )+(1-α)·SemanticSim(d i ,p j ); where α is the balance coefficient, ranging from [0,1], which is used to adjust the relative importance of association weight and semantic similarity; ClinicalWeight(d i ,p j ) is the association weight matrix calculated based on the domain knowledge graph; SemanticSim(d i ,p j ) is the semantic correlation between feature 1 and feature 2 texts calculated using a semantic similarity algorithm, ensuring that the model effectively integrates domain knowledge and data semantic information to improve the expressiveness and accuracy of the recommendation model; (4) Generate the final model input: After combining the above three processing methods for the input sequence, use the Transformer encoder to process the input sequence; add the outputs of the token embedding, segment embedding, and correlation embedding layers, and then pass them through the Transformer encoding layer to obtain the input of the final model that can comprehensively and accurately reflect the association of clinical information, providing effective data features for subsequent drug recommendation tasks.

4. The knowledge-enhanced drug recommendation method based on prior knowledge and safety perception constraints according to claim 1 is characterized in that: In step (3), model pre-training includes: (1) ICD sequence prediction ISP: By predicting whether the ICD code sequences of feature one and feature two belong to the same data feature, the model can better understand the contextual dependencies between different clinical codes and improve its ability to capture potential connections in clinical conditions; randomly replace one or more ICD codes in the feature one or feature two sequences from different samples to construct negative sample sequences; the real sample sequence and the negative sample sequence are represented as (di, pj), and the binary cross entropy loss (BCE) is selected as the loss function for training, and the formula is: in, is a two-dimensional vector, representing the probability of predicting the true sample sequence and the negative sample sequence respectively; r i is the data feature; W1 is the weight matrix of the fully connected layer, b1 is the bias vector, both of which are learnable parameters; y is the true label, if it is a true sample sequence, y = 1, if it is a negative sample sequence, y = 0; and They are Two elements in ; (2) Mask sequence reconstruction MSR: After the ISP pre-training task is completed, the MSR pre-training task is performed. This task requires the model to learn the semantic information and contextual relationship of the ICD code in the data feature representation; randomly select some ICD codes in the input sequence for masking operation, and replace the selected codes with special mask tags [MASK] with a certain random probability; encode the masked sequence through the Transformer encoder to obtain the feature representation of each position; for the masked position, its feature representation is input into a fully connected layer for prediction, and the probability distribution of each ICD code category at the mask position is output; the cross entropy loss is used to calculate the difference between the code distribution predicted by the model and the real code distribution, and the formula is in, is the predicted probability distribution; r mask is the feature of the masked position; W2 and b2 are the learnable weight and bias parameters; M is the set of masked positions; s i,c is the true label of the c-th ICD code at the i-th masked position; C is the number of categories of ICD codes; is the probability of the cth class code at the ith masked position predicted by the model.

5. The knowledge-enhanced drug recommendation method based on prior knowledge and safety perception constraints according to claim 1 is characterized in that: In step (4), model fine-tuning includes: (1) Multi-label marginal loss optimization: Use the multi-label marginal loss function to optimize the pre-trained model. The formula is: in, represents the real drug combination of the input data; is the probability of drug recommendation predicted by the model; |V m | is the number of drug types; (2) Annotation consistency constraint: Combined with multi-label marginal loss, annotation consistency constraint is designed to obtain a safe drug recommendation combination; for each input sequence, the relationship between the feature code and the drug code is δ(V d ,V m ), predict the specified drug information according to the feature encoding; map the discrete relationship to the negative logarithmic space by multiplying the t norm, and obtain the differentiable loss function of annotation consistency, which is: in, is the weight used to balance the loss during model training; V d is a feature code; V m is the drug code; V * is the set of all feature 1 code and drug code pairs; by minimizing this loss function, the model's ability to understand and predict the relationship between feature 1 and drug code is improved, maximizing the learning objective from the perspective of increasing the probability of true labels and reducing the loss of annotation consistency; (3) Taboo constraint: Based on the annotation consistency constraint, the taboo constraint is combined to limit the dangerous drug recommendation to the greatest extent; when the input sequence has two specified features V d1 , V d2 When taking contraindications, it is forbidden to take V m As a recommended drug, there is a relationship The corresponding loss function is By incorporating contraindication constraints into the loss function, the model can fully consider drug-contraindication relationships when recommending drugs, avoiding recommending drug combinations that pose safety risks. (4) Total loss calculation and adjustment: Constrain the annotation consistency loss function L A And the taboo constraint loss function L B Combined into the total loss function L constr =L A +λ*L B , where λ is a non-negative coefficient that controls each loss term; during the training process, the λ value is dynamically adjusted according to the task results, and the size of the sub-loss is reduced through iterative training, so that the parameters of the model develop in the optimal direction, thereby balancing the model's ability to learn the relationship between other features and drugs and avoid drug contraindications.

6. The knowledge-enhanced drug recommendation method based on prior knowledge and safety perception constraints according to claim 1 is characterized in that: In step (5), the recommendation result prediction includes: (1) Loss function balance: Combine the multi-label marginal loss function L by weighted sum multi And the total loss function L constr , and get the final loss function L = (1-β)L multi +β*L constr , where β is a hyperparameter that balances the contributions of different losses. By adjusting the β value, the model's performance in recommendation accuracy and safety is balanced, so that the model can accurately match the input sequence when recommending drugs while ensuring the safety of the recommended drugs. (2) Drug prediction and recommendation: After the model is trained and fine-tuned through the loss function, an MLP layer is integrated, and the output sequence of the fine-tuned model is used as the input of the MLP layer to predict the probability of each drug being recommended; the recommendation threshold is set to 0.5, and drugs with a predicted probability greater than the threshold of 0.5 are included in the final recommended drug set, that is, Provide accurate and safe drug recommendation solutions.

Images