An International Classification of Diseases Method Based on Contrastive Learning and Hyperbolic Graph Convolutional Networks

Abstract

This application is applicable to the field of data processing technology and provides an International Classification of Diseases method based on contrastive learning and hyperbolic graph convolutional networks, including: embedding each word in a clinical document into a vector space with P dimensions to obtain an original document representation; encoding the disease label using HGCN to obtain a disease label representation; processing the original document representation and the disease label representation using an attention mechanism to obtain a positive sample; inputting the original document representation and the positive sample into a convolutional attention network for encoding to obtain an attention representation of the two; performing contrastive learning on the attention representations of the two and calculating the contrastive loss; inputting the attention representations of the two into a multi-label classifier for classification and calculating the multi-label classification loss; optimizing the multi-label classifier and the contrastive learning framework using the contrastive loss and the multi-label classification loss to obtain an optimized multi-label classification result. This application can improve the efficiency and accuracy of the International Classification of Diseases.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular relates to an international disease classification method based on contrastive learning and hyperbolic graph convolutional networks. Background Art

[0002] The International Classification of Diseases (ICD), a disease classification standard developed by the World Health Organization, has been widely used in health monitoring, medical data analysis, and medical reimbursement pricing. The ICD is also an effective standard and foundation for smart healthcare applications. It culminates in individual codes describing the cause, pathology, symptoms, and anatomical location of a disease.

[0003] The International Classification of Diseases (ICD) typically requires clinicians to manually assign ICD codes to clinical documents, which is time-consuming, labor-intensive, and error-prone. Consequently, numerous studies have introduced methods for automated ICD coding. Key challenges in automated ICD coding include imbalanced label distribution, code hierarchies, and noisy text.

[0004] Unbalanced label distribution: The labels in the MIMIC-III dataset are imbalanced and exhibit a long-tail distribution, with a large number of data in the top categories and a small number of data in the tail categories. This is primarily due to the rare nature of some disease types. Some labels appear frequently, while most labels have only a small number of data points and never appear at all, resulting in a long-tail distribution of International Classification of Disease codes. In the MIMIC-III dataset, nearly 50% of codes appear only one to five times. Improving the accuracy of prediction for both dense and sparse data is particularly important. A key challenge lies in accounting for uncommon labels in classification problems.

[0005] Code Hierarchy: A tree-like hierarchy is observed in the ICD coding system, where upper-level nodes correspond to broader disease categories, while lower-level nodes correspond to more specific diseases. On the one hand, this hierarchical structure can explain why some codes are mutually exclusive. Sibling nodes in the last level of a shared parent node cannot be designated as both parent and child nodes. On the other hand, codes with smaller distances are more related. If the distance between code A and B in the code tree is smaller than the distance between B and C, then A is more likely to be the correct code in the ICD prediction than C.

[0006] Noisy text: Physicians with diverse writing styles often use abbreviations and synonyms when writing diagnostic descriptions. This can lead to potential errors and ambiguities when professional coders assign ICD codes to medical descriptions. Statistics show that only 10% of words in clinical documents are relevant to ICD codes. This creates a significant amount of redundant or misleading information, potentially negatively impacting automated ICD coding. Addressing this issue is crucial.

[0007] Currently, some consider using code hierarchies and descriptions to better represent labels to address the classification problem of imbalanced data. A common approach is to encode the text and labels separately and then combine them using a feature aggregation layer before classification. While these methods can fully combine document representations with label features to enhance the representation, which is useful for label classification, label encoders provide a completely constant and identical label representation for any input text representation based on the label description or hierarchy. This interaction of text only with a constant hierarchical representation is unnecessary and inefficient. On the other hand, most models utilize classification hierarchies through graph encoders, such as graph convolutional neural networks (GCNNs) and graph generalized recurrent neural networks (GRNNs). These models in graph neural networks are limited by Euclidean geometry representations and result in significant distortion of the hierarchical structure, resulting in low accuracy for the International Classification of Diseases. Summary of the Invention

[0008] The embodiment of the present application provides an international disease classification method based on contrastive learning and hyperbolic graph convolutional network, which can solve the problems of low efficiency and accuracy of international disease classification.

[0009] The present application embodiment provides an international disease classification method based on contrastive learning and hyperbolic graph convolutional networks, including:

[0010] Embed each word in the clinical document into a vector space with P dimensions to obtain the original document representation;

[0011] Use hyperbolic graph convolutional neural network to encode disease labels and obtain disease label representation;

[0012] The original document representation and disease label representation are processed using the attention mechanism to obtain a document representation with a hierarchical relationship injected with disease labels. The document representation with a hierarchical relationship injected with disease labels is used as a positive sample in contrastive learning.

[0013] The original document representation and positive sample are input into the convolutional attention network for encoding respectively, and the attention representation based on the original document representation and the attention representation based on the positive sample are obtained;

[0014] Input the attention representation based on the original document representation and the attention representation based on the positive sample into the contrastive learning framework for contrastive learning, and calculate the contrast loss between the attention representation based on the original document representation and the attention based on the positive sample;

[0015] Input the attention representation based on the original document representation and the attention representation based on the positive sample into the multi-label classifier for classification, and calculate the multi-label classification loss of the original document representation and the multi-label classification loss of the positive sample;

[0016] The total model loss is calculated using the contrastive loss, the multi-label classification loss of the original document representation, and the multi-label classification loss of the positive samples. The total model loss is then used to optimize the parameters of the multi-label classifier and the contrastive learning framework until the total model loss reaches the preset loss value.

[0017] The classification result represented by the original document when the total loss of the model reaches the preset loss value is used as the multi-label classification result of the clinical document.

[0018] Optionally, each word in the clinical document is embedded into a vector space with P dimensions to obtain the original document representation, including:

[0019] Each word in the clinical document is embedded into a vector space with P dimensions using a pre-trained word embedding model to obtain the original document representation; the original document is represented as a matrix X;

[0020] X={x1,…,x N}

[0021] Among them, x N represents the embedding vector corresponding to the Nth word in the clinical document, and N represents the total number of words in the clinical document.

[0022] Optionally, a hyperbolic graph convolutional neural network is used to encode the disease label to obtain a disease label representation, including:

[0023] Use the pre-trained word embedding model to embed each disease label into a vector space with P dimensions to obtain the disease label embedding representation C;

[0024] The disease label embedding representation C is encoded using a hyperbolic graph convolutional neural network to obtain the disease label representation G.

[0025] Optionally, the attention representation A based on the original document representation is:

[0026] A={A1,…,A j}

[0027] A j represents the jth hidden representation in A, where j is the total number of hidden representations in A;

[0028] Attention representation based on positive samples for:

[0029]

[0030] Indicates A j The corresponding positive samples.

[0031] Optionally, the attention representation based on the original document representation and the attention representation based on the positive sample are input into the contrastive learning framework for contrastive learning, and the contrastive loss of the attention representation based on the original document representation and the attention representation based on the positive sample is calculated, including:

[0032] The hidden representation A in the attention representation A based on the original document representation i and A i The corresponding positive sample As a positive example, the positive example is input into the contrastive learning framework for contrastive learning, i = 1, ..., j;

[0033] Calculate the hidden representation A i and positive samples Contrast loss;

[0034] The average of the contrastive losses corresponding to all positive examples is taken as the contrastive loss between the attention representation based on the original document representation and the attention based on the positive samples.

[0035] Optionally, compute the hidden representation A i and positive samples Contrastive loss, including:

[0036] By formula Calculate the hidden representation A i and positive samples Contrastive loss

[0037] Among them, the sim function represents the calculation of cosine similarity, Z i =W S ReLu(W c A i ), W S ∈R |L|×D , W c ∈R D×D , |L| represents the total number of disease labels, D represents the output dimension of the ReLu output layer, τ represents the temperature hyperparameter, Z k Represents Z i and A collection of .

[0038] Optionally, calculate the multi-label classification loss of the original document representation and the multi-label classification loss of the positive samples, including:

[0039] By formula Calculate the multi-label classification loss L for the original document representation M ; |L| represents the total number of disease labels, y mRepresents the classification result of the mth disease label output by the multi-label classifier when classifying the original document representation. W o represents the weight, W o ∈R D , D represents the sigmoid output layer dimension, b o express deviation;

[0040] By formula Calculate the multi-label classification loss L^ for positive samples M ; Represents the classification result of the mth disease label output by the multi-label classifier when classifying the positive sample,

[0041] Optionally, the total model loss is calculated using the contrastive loss, the multi-label classification loss of the original document representation, and the multi-label classification loss of the positive samples, including:

[0042] By the formula L=L M +L^ M +βL con Calculate the total loss of the model L; L M represents the multi-label classification loss of the original document representation, L^ M represents the multi-label classification loss of positive samples, L con represents the contrastive loss between the attention representation based on the original document representation and the attention based on the positive sample, and β represents the weight of the contrastive loss.

[0043] The above solution of the present application has the following beneficial effects:

[0044] In an embodiment of the present application, a hyperbolic graph convolutional neural network is used to encode disease labels to capture the code hierarchy and relationship on the invisible graph without distortion, thereby greatly improving the accuracy of the International Classification of Diseases. In addition, the original document representation and disease label representation of the clinical document are processed by an attention mechanism to generate hierarchical perception positive samples, instead of aggregating text and constant code features, introducing contrastive learning of automatic International Classification of Diseases coding, and optimizing the multi-label classifier and contrastive learning framework using contrastive learning loss and multi-label classification loss to obtain optimized multi-label classification results. This method eliminates the unnecessary process of text interacting only with a constant hierarchical representation, greatly improving the efficiency of the International Classification of Diseases.

[0045] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 A flowchart of an international disease classification method based on contrastive learning and hyperbolic graph convolutional networks provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0049] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0050] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0051] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0052] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0054] In response to the current low efficiency and accuracy of the International Classification of Diseases, an embodiment of the present application provides an International Classification of Diseases method based on contrastive learning and a hyperbolic graph convolutional network. This method uses a hyperbolic graph convolutional neural network to encode disease labels to capture the code hierarchy and relationship on the invisible graph without distortion, thereby greatly improving the accuracy of the International Classification of Diseases. In addition, the original document representation and disease label representation of the clinical document are processed by an attention mechanism to generate hierarchical perception positive samples, instead of aggregating text and constant code features. Automatic contrastive learning of International Classification of Diseases coding is introduced, and contrastive learning loss and multi-label classification loss are used to optimize the multi-label classifier and contrastive learning framework to obtain an optimized multi-label classification result. This process of omitting the process of text interacting only with a constant hierarchical representation greatly improves the efficiency of the International Classification of Diseases.

[0055] The following is an illustrative example of the International Classification of Diseases method based on contrastive learning and hyperbolic graph convolutional network provided in the embodiments of the present application, with reference to specific examples.

[0056] like Figure 1 As shown, the International Classification of Diseases method based on contrastive learning and hyperbolic graph convolutional network provided in the embodiment of the present application includes the following steps:

[0057] In step 11, each word in the clinical document is embedded into a vector space with P dimensions to obtain the original document representation.

[0058] The clinical document may be a patient's medical history, which contains information such as the patient's disease diagnosis and treatment history. The vector space is low-dimensional to improve the efficiency of the International Classification of Diseases. It is understood that the specific dimension P of the vector space can be set based on actual conditions, for example, to 100.

[0059] In some embodiments of the present application, a pre-trained word embedding model can be used to embed each word in a clinical document into a vector space with P dimensions to obtain the original document representation.

[0060] The above word embedding model can adopt a commonly used word embedding model (such as word2vec model), and the original document representation obtained can be specifically a matrix X, X = {x1,…,x N}. Among them, x N represents the embedding vector corresponding to the Nth word in the clinical document, and N represents the total number of words in the clinical document.

[0061] Step 12: Encode the disease label using a hyperbolic graph convolutional neural network to obtain a disease label representation.

[0062] The above disease labels can be understood as all labels corresponding to international diseases, such as heart disease, malignant tumors, heart bypass surgery, etc.

[0063] In some embodiments of the present application, a pre-trained word embedding model can be used to embed each disease label into a vector space with P dimensions to obtain a disease label embedding representation C, and then a hyperbolic graph convolutional neural network (HGCN) is used to encode the disease label embedding representation C to obtain a disease label representation G. Where C = {c1,…,c |L|}, c |L| Represents the embedding vector of the |L|th disease label; G={g1,…,g |L|}, g |L| Indicates that HGCN is |L| The disease labels obtained by encoding are represented, and |L| represents the total number of disease labels.

[0064] That is, in some embodiments of the present application, shared embedding weights are used for clinical documents and disease labels, and all label embeddings are represented as C = {c1, ..., c |L|}, which has a learnable embedding of size P, and uses a 2-layer HGCN to encode the label embedding representation C to obtain G = {g1,…,g |L|}.

[0065] It is worth mentioning that, considering the hierarchical structure of disease labels, a hyperbolic graph convolutional neural network is used as a graph encoder to encode and obtain disease label representation, which can capture the code hierarchy and relationships on invisible graphs without distortion, thereby facilitating the improvement of the accuracy of the International Classification of Diseases.

[0066] In step 13, the original document representation and the disease label representation are processed using the attention mechanism to obtain a document representation of a hierarchical relationship injected with disease labels, and the document representation of a hierarchical relationship injected with disease labels is used as a positive sample in comparative learning.

[0067] In some embodiments of the present application, scaled dot product attention can be used to process the original document representation and the disease label representation to obtain a document representation that incorporates the hierarchical relationship of the disease labels. Specifically, the original document representation and the disease label representation can be input into a convolutional layer for processing, and the processing results of the convolutional layer are then input into an attention layer for processing to obtain a document representation that incorporates the hierarchical relationship of the disease labels.

[0068] It is worth mentioning that the original document representation and disease label representation of clinical documents are processed through the attention mechanism to generate hierarchically perceived positive samples, instead of aggregating text and constant code features. This eliminates the unnecessary process of text interacting only with constant hierarchical representations, greatly improving the efficiency of the International Classification of Diseases.

[0069] In step 14, the original document representation and the positive sample are respectively input into the convolutional attention network for encoding to obtain the attention representation based on the original document representation and the attention representation based on the positive sample.

[0070] The above convolutional attention network can adopt the commonly used convolutional attention network structure. The convolutional attention network encodes the original document representation to obtain the attention representation A based on the original document representation: A={A1,…,A j}, A j Represents the jth hidden representation in A, where j is the total number of hidden representations in A; the positive sample-based attention representation obtained by the convolutional attention network encoding the positive sample for: Indicates A j The corresponding positive samples.

[0071] In step 15, the attention representation based on the original document representation and the attention representation based on the positive sample are input into the contrastive learning framework for contrastive learning, and the contrast loss of the attention representation based on the original document representation and the attention representation based on the positive sample is calculated.

[0072] In some embodiments of the present application, the hidden representation in the attention representation A based on the original document representation and the corresponding positive sample can be used as a positive example, and each positive example can be input into the contrastive learning framework for contrastive learning in turn. Finally, the contrastive loss of the attention representation based on the original document representation and the contrastive loss of the attention based on the positive sample are obtained based on the contrastive loss of each positive example, thereby solving the problems related to the imbalanced distribution of disease labels, the hierarchy of disease labels and noisy text.

[0073] In step 16, the attention representation based on the original document representation and the attention representation based on the positive sample are input into the multi-label classifier for classification, and the multi-label classification loss of the original document representation and the multi-label classification loss of the positive sample are calculated.

[0074] The multi-label classifier is mainly used for label classification, and can specifically adopt a commonly used multi-label classifier structure.

[0075] In some embodiments of the present application, the multi-label classification loss L represented by the original document can be calculated by the following formula: M :

[0076]

[0077] Among them, |L| represents the total number of disease labels, y m represents the classification result of the mth disease label output by the multi-label classifier when classifying the original document representation (it should be noted that this classification result refers to the classification result obtained when the attention representation based on the original document representation is input into the multi-label classifier for classification). W o represents the weight, W o ∈R D , D represents the dimension of the sigmoid output layer, b o Indicates deviation.

[0078] In some embodiments of the present application, the multi-label classification loss L^ of the positive sample can be calculated by the following formula: M :

[0079]

[0080] in, represents the classification result of the mth disease label output by the multi-label classifier when classifying the positive sample (it should be noted that the classification result refers to the classification result obtained when the attention representation based on the positive sample is input into the multi-label classifier for classification).

[0081] In step 17, the total model loss is calculated using the contrastive loss, the multi-label classification loss of the original document representation, and the multi-label classification loss of the positive sample, and the total model loss is used to optimize the parameters of the multi-label classifier and the contrastive learning framework until the total model loss reaches the preset loss value.

[0082] In some embodiments of the present application, the formula L=L M +L^ M +βL con Calculate the total loss L of the model. Among them, L M represents the multi-label classification loss of the original document representation, L^ M represents the multi-label classification loss of positive samples, L conrepresents the contrast loss between the attention representation based on the original document representation and the attention based on the positive sample, β represents the weight of the contrast loss, and β is a hyperparameter that can be set according to actual conditions.

[0083] It is understandable that the above total model loss is mainly used to iteratively optimize the parameters of the multi-label classifier and contrastive learning framework. The optimization goal is to minimize the total model loss, even if the total model loss is less than or equal to the above preset loss value.

[0084] In step 18, the classification result represented by the original document when the total loss of the model reaches a preset loss value is used as the multi-label classification result of the clinical document.

[0085] It is understood that when the total loss of the above model is minimized, the classification accuracy of the above multi-label classifier is optimal. Therefore, the classification result at this time can be used as the multi-label classification result of the clinical document. In practical applications, for a clinical document, the multi-label classification result of the clinical document may be: heart disease and heart bypass surgery.

[0086] It is worth mentioning that this application solves the problems of unbalanced label distribution, label hierarchy and noisy text by introducing a hyperbolic graph convolutional neural network with contrastive learning (HGCN-CL); a hyperbolic graph convolutional neural network is used on ICD codes to capture the code hierarchy and relationship on invisible graphs without distortion. In addition, the original document representation and disease label representation of clinical documents are processed by an attention mechanism to generate hierarchical-aware positive samples instead of aggregating text and constant code features. Contrastive learning of automatic International Classification of Diseases codes is introduced, and contrastive learning loss and multi-label classification loss are used to optimize the multi-label classifier and contrastive learning framework to obtain optimized multi-label classification results. This method greatly improves the efficiency of the International Classification of Diseases by eliminating the unnecessary process of text interacting only with a constant hierarchical representation.

[0087] The specific implementation of contrastive learning and contrastive loss is exemplified below.

[0088] In some embodiments of the present application, the above step 15, inputting the attention representation based on the original document representation and the attention representation based on the positive sample into the contrastive learning framework for contrastive learning, and calculating the contrastive loss of the attention representation based on the original document representation and the attention representation based on the positive sample, includes the following steps:

[0089] Step 15.1: The hidden representation A in the attention representation A based on the original document representation is i and A i The corresponding positive sample As a positive example, the positive example is input into the contrastive learning framework for contrastive learning, i = 1, ..., j;

[0090] Step 15.2, calculate the hidden representation A i and positive samples Contrast loss;

[0091] In step 15.3, the average of the contrastive losses corresponding to all positive examples is taken as the contrastive loss between the attention representation based on the original document representation and the attention based on the positive samples.

[0092] Specifically, the formula Calculate the hidden representation A i and positive samples Contrastive loss The sim function represents the calculation of cosine similarity, Z i =W S ReLu(W c A i ), W S ∈R |L|×D , W c ∈R D×D , |L| represents the total number of disease labels, D represents the dimension of the ReLu output layer, τ represents the temperature hyperparameter, Z k Represents Z i and A collection of .

[0093] The following is an illustrative example of the International Classification of Diseases method based on contrastive learning and hyperbolic graph convolutional networks.

[0094] In this example, we implemented this model using PyTorch 1.3 (a deep learning framework based on the Python programming language) and trained it using the standard Adam optimizer (an adaptive optimization algorithm). During preprocessing, we obtained 100-dimensional word embeddings by pre-training the word2vec model on the MIMIC-III and MIMIC-II datasets (as shown in Table 1).

[0095]

[0096] Table 1

[0097] The dropout rate was set to 0.2, and the convolution filter size was set to 10. Other hyperparameters were selected through a random search process to find the optimal values, including the learning rate to 0.0001 and the batch size to 12. In addition, the loss threshold was set to 0.05, and the loss weight was set from 0 to 0.5. The temperature of the comparison module was set to 1. If the top 8 accuracy did not show any improvement, the training process was programmed to end early. After training stopped, the top 8 scoring models were selected and applied to the test set.

[0098] To further evaluate the performance of this application, it is also compared with previous state-of-the-art models and few-shot learning methods for automatic ICD coding using a code hierarchy and taking into account typical evaluations of multi-label classification algorithms, using evaluation metrics such as micro-average and macro-average F1 scores (F1 score is an indicator used to measure the accuracy of a binary classification model), micro-average and macro-average AUC (AUC is the area under the receiver operating characteristic curve (ROC curve)). Micro-average and macro-average F1 scores for diagnosis and procedure codes are also provided so that the results can be compared with earlier studies. In addition, the top k (P@k) precision is reported as a metric because Macro F1 (Macro F1 is the arithmetic mean of the F1 scores for each class) may be significantly affected by highly discriminative classes (i.e., high recall, high precision), but is not easily affected by data imbalance. P@k calculates the scores of the top k highest-scoring labels that are accurately predicted. P@8 and P@15 were selected for the MIMIC-III full dataset (MIMIC-III full), P@5 was selected for the MIMIC-III 50 top 50 dataset (MIMIC-III TOP 50), and P@8 was selected for the MIMIC-III TOP 50.

[0099] To ensure a fair comparison, the same pre-trained model word2vec was used for word embedding, and each model was run ten times with different random seeds for initialization. The analysis focuses on the performance of the model on the MIMIC-III full dataset and the MIMIC-III TOP 50 frequently occurring code dataset. The evaluation results are listed in Tables 2 and 3. The performance of the model on the MIMIC-II dataset is also compared, as shown in Table 4. Among them, Bi-GRU is a neural network model for sequence modeling, CAML is a convolutional attention multi-label classification network, DR-CAML is a label regularized convolutional attention multi-label classification network, MSATT-KG is a knowledge graph injected multi-label classification network, HyperCore is an ICD automatic encoding based on hyperbolic space and co-occurrence graph representation, HGCN-CL is a hyperbolic graph convolutional neural network for contrastive learning provided in the embodiment of the present application; Micro refers to micro, Macro refers to macro, Diag refers to diagnosis code label, and Proc refers to step code label.

[0100]

[0101] Table 2

[0102]

[0103]

[0104] Table 3

[0105]

[0106] Table 4

[0107] The model of this application outperforms all baselines on all evaluation metrics of MIMIC-III full. Specifically, HGCN-CL shows 2.7% and 1.2% improvements in Micro-F1 and Macro-F1 compared to HyperCore, as shown in Table 2. These results demonstrate the success of HGCN-CL. More importantly, HGCN-CL has achieved significant improvements over CAML and DR-CAML, which shows the importance of contrastive learning for HGCN-CL. This is because contrastive learning can improve the classification accuracy of rare labels by learning the similarities and differences between labels in multi-label classification. It also shows how HGCN with dynamic code interactions can capture more complex code interaction information than using consistent code embeddings.

[0108] As can be seen from Table 2, Macro-F1 grows faster than Micro-F1. This is because Macro-F1 focuses more on infrequent code prediction, while HGCN-CL has a great advantage in rare label prediction.

[0109] For MIMIC-III TOP 50, HGCN-CL still performs poorly in the presence of most state-of-the-art models, as can be seen in Table 3. Micro-F1 and Macro-F1 improve by 4.5% and 4.4% respectively compared to the previous top model MSATT-KG. Notably, the most significant improvement is achieved in the top 5 (P@5) accuracy, increasing from 64.4% to 71.0% compared to MSATT-KG. This finding emphasizes the effectiveness of hierarchical labeling and contrastive learning, not only for infrequent labels, but also for frequent labels, finding that labels that often co-occur have higher similarities, thus giving them higher probabilities to predict.

[0110] Table 4 shows the performance evaluation results of our HGCN-CL, which outperforms the HyperCore model in Micro F1, Macro F1, and P@8, with values of 0.513, 0.076, and 0.575, respectively. In particular, the HGCN-CL model significantly outperforms other methods on MIMIC-II, achieving a 3.6% improvement over HyperCore in Micro F1. Furthermore, we observed significantly better performance on MIMIC-III than on MIMIC-II across all evaluation metrics. This improvement may be attributed to the larger sample size of MIMIC-III.

[0111] The innovations of HGCN-CL in this application mainly include three parts: HGCN, positive sample generation, and contrastive learning. In order to verify the contribution of these three parts to the success of our proposed model, ablation experiments were performed by replacing or deleting them in the network of this application. The ablation experiment was performed using MIMIC-III as an example because it has a larger and more representative dataset. The results are shown in Table 5. The effects of HGCN graph encoder, positive sample generation, and contrast are further analyzed in detail. Among them, -rpGCN is the performance of removing the graph convolutional network, -rpHyperbolic Embedding is the performance of removing hyperbolic embedding, -rmgraph encoder is the performance of removing the graph encoder, -rpWord deletion is the performance of removing word deletion in positive sample generation, -rpDropout is the performance of removing Dropout, and -rmcontrastive loss is the performance of removing contrastive learning loss.

[0112]

[0113] Table 5

[0114] It can be seen from the above examples that the international disease classification method provided in the embodiments of the present application can significantly improve the accuracy of the international disease classification.

[0115] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for international classification of diseases based on contrastive learning and hyperbolic graph convolutional networks, characterized in that: include: Embed each word in the clinical document into a vector space with P dimensions to obtain the original document representation; Use hyperbolic graph convolutional neural network to encode disease labels and obtain disease label representation; The original document representation and the disease label representation are processed using an attention mechanism to obtain a document representation of a hierarchical relationship injected with disease labels, and the document representation of the hierarchical relationship injected with disease labels is used as a positive sample in contrastive learning; Inputting the original document representation and the positive sample into a convolutional attention network for encoding respectively, to obtain an attention representation based on the original document representation and an attention representation based on the positive sample; Inputting the attention representation based on the original document representation and the attention representation based on the positive sample into a contrastive learning framework for contrastive learning, and calculating the contrastive loss of the attention representation based on the original document representation and the attention representation based on the positive sample; Inputting the attention representation based on the original document representation and the attention representation based on the positive sample into a multi-label classifier for classification, and calculating the multi-label classification loss of the original document representation and the multi-label classification loss of the positive sample; Calculating a total model loss using the contrastive loss, the multi-label classification loss of the original document representation, and the multi-label classification loss of the positive sample, and optimizing the parameters of the multi-label classifier and the contrastive learning framework using the total model loss until the total model loss reaches a preset loss value; The classification result represented by the original document when the total loss of the model reaches a preset loss value is used as the multi-label classification result of the clinical document.

2. The International Classification of Diseases method according to claim 1, characterized in that: The method of embedding each word in the clinical document into a vector space with P dimensions to obtain the original document representation includes: Embed each word into a vector space with P dimensions using a pre-trained word embedding model to obtain an original document representation; the original document representation is a matrix X; X={x1,…,x N } Among them, x N represents the embedded word vector corresponding to the Nth word in the clinical document, and N represents the total number of words in the clinical document.

3. The International Classification of Diseases method according to claim 1, characterized in that: The method of encoding the disease label using the hyperbolic graph convolutional neural network to obtain the disease label representation includes: Use the pre-trained word embedding model to embed each disease label in the clinical document into a vector space with P dimensions to obtain the disease label embedding representation C; The disease label embedding representation C is encoded using a hyperbolic graph convolutional neural network to obtain a disease label representation G.

4. The International Classification of Diseases method according to claim 1, wherein: The attention representation A based on the original document representation is: A={A1,…,A j } A j represents the jth hidden representation in A, where j is the total number of hidden representations in A; Attention representation based on the positive sample for: Indicates A j The corresponding positive samples.

5. The International Classification of Diseases method according to claim 4, characterized in that: The inputting the attention representation based on the original document representation and the attention representation based on the positive sample into a contrastive learning framework for contrastive learning, and calculating the contrastive loss of the attention representation based on the original document representation and the attention representation based on the positive sample, includes: The hidden representation A in the attention representation A based on the original document representation i and A i The corresponding positive sample As a positive example, the positive example is input into the contrastive learning framework for contrastive learning, i=1,…,j; Calculate the hidden representation A i and positive samples Contrast loss; The average of the contrastive losses corresponding to all positive examples is taken as the contrastive loss between the attention representation based on the original document representation and the attention based on the positive sample.

6. The International Classification of Diseases method according to claim 5, characterized in that: The calculation hidden representation A i and positive samples Contrastive loss, including: By formula Calculate the hidden representation A i and positive samples Contrastive loss Among them, the sim function represents the calculation of cosine similarity, Z i =W S ReLu(W c A i ), W S ∈R |L|×D , W c ∈R D×D , |L| represents the total number of disease labels, D represents the dimension of the ReLu output layer, τ represents the temperature hyperparameter, Z k Represents Z i and A collection of .

7. The International Classification of Diseases method according to claim 1, characterized in that: The calculating the multi-label classification loss of the original document representation and the multi-label classification loss of the positive sample includes: By formula Calculate the multi-label classification loss L for the original document representation M ; |L| represents the total number of disease labels, y m represents the classification result of the mth disease label output by the multi-label classifier when classifying the original document representation, W o represents the weight, W o ∈R D , D represents the dimension of the sigmoid output layer, b o express deviation; By formula Calculate the multi-label classification loss L^ of the positive sample M ; represents the classification result of the mth disease label output by the multi-label classifier when classifying the positive sample, 8. The International Classification of Diseases method according to claim 1, characterized in that: The calculating the total model loss by using the contrast loss, the multi-label classification loss of the original document representation, and the multi-label classification loss of the positive sample includes: By the formula L=L M +L^ M +βL con Calculate the total loss of the model L; L M represents the multi-label classification loss of the original document representation, L^ M represents the multi-label classification loss of the positive sample, L con represents the contrastive loss between the attention representation based on the original document representation and the attention based on the positive sample, and β represents the weight of the contrastive loss.

Images