An automatic coding method for the International Classification of Diseases

Abstract

This invention discloses a method for automatically encoding the International Classification of Diseases. The method involves obtaining electronic medical record data from a medical database and performing preliminary data cleaning. The method then extracts electronic medical record text and associates it with the ICD codes used in doctors' diagnoses to construct a training dataset. A multi-faceted knowledge base is constructed, including a UMLS synonym library, an unstructured knowledge base, and an auxiliary knowledge base generated using LLM. An ICD encoding model is constructed, including a context encoding module, an auxiliary knowledge screening module, a multi-attention interaction module, and a fusion prediction module. The method then optimizes the ICD encoding model using a binary cross-entropy loss function to achieve automatic encoding of the International Classification of Diseases. The method effectively captures the multi-level semantic interactions between text and code, improving the ability to distinguish semantic information at different levels in complex text and encoding accuracy. It also enables more accurate extraction of key information, improving encoding quality in complex cases and ambiguous diagnosis scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to an automatic coding method for realizing the International Classification of Diseases. Background Art

[0002] This invention relates to the field of natural language processing (NLP), specifically to methods for automatically encoding medical information based on deep learning models. Specifically, it addresses a system and method for automatically encoding the International Classification of Diseases (ICD) using multi-source knowledge and a multi-level attention mechanism. With the widespread adoption of electronic health records (EHRs), medical institutions have accumulated a large amount of unstructured or semi-structured clinical text data, such as medical records, diagnostic reports, and discharge summaries. This data contains rich diagnostic information, and there is an urgent need to automatically map it to ICD codes to support medical record management, disease statistical analysis, and medical insurance reimbursement. The task of automatically encoding ICDs has become a cutting-edge challenge in NLP research due to its vast code system (for example, the ICD-10 contains approximately 70,000 codes) and the complexity of clinical text (e.g., its length and dense use of specialized terminology).

[0003] In recent years, deep learning technologies, particularly pre-trained language models based on the Transformer architecture (such as BERT and RoBERTa), have made significant progress in the field of automatic ICD encoding. Pre-trained on large-scale corpora, these models are able to capture the deep semantic information of text and, combined with downstream fine-tuning for tasks, achieve encoding functionality. However, existing technologies still have limitations in handling semantic gaps, long-tail label distributions (i.e., rare codes), and multi-label co-occurrence relationships. In particular, they are insufficient in encoding complex clinical cases and ambiguous diagnoses. For example, one study improved encoding accuracy through label attention mechanisms. However, relying solely on a single attention mechanism, it did not integrate external knowledge or model semantic dependencies between codes, limiting its ability to handle rare codes and complex cases. Another study proposed an automatic ICD encoding method based on a pre-trained language model (PLM). The ClinicalBERT model was specifically optimized for processing clinical notes and fine-tuned to achieve a mapping from text to ICD codes. However, it did not incorporate multi-source knowledge and had limited ability to model long-tail label distributions and inter-code relationships. Another study proposed the Multiple Synonyms Matching Network (MSMN), which enhances the robustness of coding by matching synonyms of ICD codes with expressions in clinical text. This method focuses on semantic similarity between codes, but it does not fully utilize external knowledge or multi-level attention mechanisms, limiting its performance for rare codes and multi-label prediction.

[0004] These technologies have the following shortcomings: 1) They fail to systematically integrate multi-source auxiliary knowledge (such as UMLS, unstructured knowledge, and LLMs), resulting in limited semantic enhancement capabilities. 2) Single or simple attention mechanisms or matching strategies make it difficult to simultaneously capture global dependencies and fine-grained interactions, limiting the ability to handle complex cases. 3) Insufficient modeling of long-tail label distributions and semantic dependencies between codes affects the accuracy of rare code and multi-label predictions. Based on these technical backgrounds and shortcomings, the present invention proposes an innovative KLaCH framework that significantly improves the performance of ICD automatic encoding by uniformly integrating multi-source knowledge and a collaborative attention mechanism. Summary of the Invention

[0005] Purpose of the invention: The present invention proposes an automatic encoding method for the International Classification of Diseases, which significantly improves the performance of ICD automatic encoding by uniformly integrating multi-source knowledge and a collaborative attention mechanism.

[0006] Technical solution: The method for automatically encoding the International Classification of Diseases described in the present invention specifically includes the following steps:

[0007] (1) Obtain electronic medical record data from the medical database and perform preliminary data cleaning;

[0008] (2) Extract electronic medical record text, associate it with the ICD code of the doctor's diagnosis, and construct a training dataset;

[0009] (3) Constructing a multi-knowledge base, including the UMLS synonym database, unstructured knowledge base, and auxiliary knowledge base generated by LLM;

[0010] (4) Construct an ICD encoding model, including a context encoding module, an auxiliary knowledge screening module, a multiple attention interaction module, and a fusion prediction module;

[0011] (5) The binary cross entropy loss function is used to optimize the ICD coding model, output the probability of each ICD code, and realize automatic prediction of the ICD code of the electronic medical record text.

[0012] Furthermore, it is characterized in that the implementation process of step (2) is as follows:

[0013] For each patient's electronic medical record, the discharge summary section is selected as the electronic medical record text, and the extracted electronic medical record text is associated with the ICD code of the doctor's diagnosis to form training data; the electronic medical record texts of all patients and their corresponding ICD codes are summarized to form an ICD automatic coding training dataset; all ICD codes appearing in the training dataset are counted, and the ICD code of each training sample is summarized as the training set label.

[0014] Furthermore, the implementation process of step (3) is as follows:

[0015] UMLS Thesaurus: Extracts a collection of synonymous terms by aligning ICD codes with UMLS Concept Unique Identifiers (CUIs) ;

[0016] Unstructured knowledge base: retrieve ICD code related entries, extract disease definitions, symptom descriptions and physiological characteristics, and construct structured knowledge entries ;

[0017] Using LLM to generate auxiliary knowledge base: design prompt templates and generate expanded text through Qwen large model .

[0018] Furthermore, the context encoding module in step (4) is:

[0019] Using the RoBERTa pre-trained model, we encode clinical text in blocks and generate contextual representations:

[0020] ;

[0021] in, is the word vector of the c-th text block, is the output representation, T is the number of words in the block, and d is the hidden layer dimension.

[0022] Furthermore, the auxiliary knowledge screening module in step (4) is:

[0023] Encode multi-source knowledge items into vectors:

[0024] ;

[0025] Using the maximum diversity screening algorithm MDP, we select M knowledge vectors with the widest semantic coverage. The specific problems are as follows:

[0026] ;

[0027] in, is the cosine distance, To select variables, we finally get .

[0028] Furthermore, the multi-attention interaction module in step (4) is composed of a label attention mechanism and a cross attention mechanism;

[0029] Label self-attention is to model the global dependency between ICD codes and solve the problem of sparse semantics of low-frequency labels in long-tail distribution; the input is the contextual representation of clinical text and label embedding L; through linear transformation Will Mapped to query vector Q, key vector K is directly used The value vector V is The original semantic representation of ; the attention weight α is generated by the nonlinear activation function tanh and softmax calculation:

[0030] ;

[0031] in, , is a trainable parameter; next, by applying α to the context representation , computes a label-specific contextual representation , while the document-level tag-specific representation It is generated by linearly transforming the output of each block and summing them up:

[0032] ;

[0033] in, is a learnable weight matrix, C is the total number of blocks;

[0034] The cross-attention mechanism includes label-context cross-attention and knowledge-context cross-attention. The label-context cross-attention aims to align the fine-grained semantics of ICD codes and clinical texts, and accurately locate key text fragments related to diagnostic codes; the knowledge-context cross-attention fills the semantic gap between clinical texts and diagnostic codes by introducing external multi-source knowledge.

[0035] Furthermore, the label-context cross attention input is the label embedding L and the clinical text representation ; The query vector Q directly uses L to represent the semantic requirements of the label; the key K and value V are both from , representing the contextual features of the text; the interaction representation is calculated by scaling the dot product attention:

[0036] ;

[0037] in, is a learnable weight matrix, C is the total number of blocks; output In , each label vector aggregates the text segments that are most semantically relevant to it.

[0038] Furthermore, the knowledge-context cross attention introduces external multi-source knowledge input as the filtered knowledge representation , M is the number of knowledge items, and d represents the hidden layer dimension of the model; For clinical text representation, query Q is , indicating the semantic requirements of external knowledge; the key K is , the value V is the average pooling result of the knowledge vector , to aggregate global knowledge semantics, attention is calculated as follows:

[0039] ;

[0040] Among them, CA stands for cross attention mechanism, and the final output is Injecting external knowledge semantics into textual representations.

[0041] Furthermore, the fusion prediction module fuses the labels, texts, and knowledge representations generated by the multimodal interaction module and generates the final ICD probability distribution through a lightweight convolutional network. The outputs of label self-attention, label-context cross-attention, and knowledge-context cross-attention are superimposed and spliced ​​into a unified representation:

[0042] ;

[0043] Where C is the number of text blocks, and 3d is the concatenation dimension of the three attention outputs. Then, through two-level one-dimensional convolution and nonlinear activation function, the fused representation is mapped to the probability distribution of ICD encoding:

[0044] ;

[0045] Among them, the first layer of convolution is used to extract local semantic patterns from the fusion representation, the LeakyReLU activation function is used to alleviate the gradient vanishing problem, and the second layer of convolution is used to map the intermediate features to the ICD label space to generate a multi-label probability distribution.

[0046] Furthermore, the implementation process of optimizing the ICD coding model using the binary cross entropy loss function in step (5) is as follows:

[0047] ;

[0048] in, , represents the true label of the j-th ICD code, 1 means existence, 0 means non-existence, represents the model's predicted probability for the j-th ICD code; is the total number of ICD coding labels.

[0049] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. This paper proposes a novel framework, KLaCH, to integrate multi-source information, including a unified medical language system, unstructured knowledge bases, and large language models, to enhance semantic representation capabilities and thus improve the accuracy of the model in identifying rare codes and ambiguous diagnoses in complex clinical texts.

[0051] 2. To address the shortcomings of single or simple attention mechanisms in capturing both global dependencies and fine-grained interactions, this paper introduces three novel collaborative attention mechanisms into the KLaCH framework to effectively model the global dependencies of long texts and fine-grained label-context interactions, thereby improving the processing performance of multi-label prediction tasks.

[0052] 3. This paper addresses the challenges of insufficient modeling of long-tail tag distribution and semantic dependencies between codes. By optimizing the synergy between multi-source knowledge and attention mechanisms through the KLaCH framework, it improves the prediction ability of rare codes and enhances the capture of co-occurrence relationships between ICD codes, thereby improving the accuracy of multi-label prediction.

[0053] 4. This paper addresses the problem that existing technologies perform poorly when processing lengthy, jargon-heavy, and ambiguous clinical texts. By leveraging the multi-level semantic enhancement and attention mechanism of the KLaCH framework, the model can more accurately extract key information, thereby improving the encoding quality in complex cases and ambiguous diagnostic scenarios.

[0054] 5. In response to the limitation of the existing technology that lacks a multi-level attention mechanism, the present invention effectively captures the multi-level semantic interactions between text and code through the multi-level hybrid attention mechanism in the KLaCH framework, thereby improving the model's ability to distinguish different levels of semantic information in complex text and encoding accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Flowchart of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be described in further detail below with reference to the accompanying drawings.

[0057] like Figure 1 As shown in the figure, this paper proposes an automatic encoding method for the International Classification of Diseases (ICD). The KLaCH (Knowledge, Label Attention, and Cross-Attention in Harmony) framework is used to automatically predict ICD codes from electronic medical record text. This method improves the accuracy of ICD code prediction by integrating external knowledge (knowledge) and a multi-layered attention mechanism, combined with code dependency modeling (CodeHierarchy).

[0058] Step 1: Obtain electronic medical record data from the medical database and perform preliminary data cleaning.

[0059] Obtain electronic medical record data from medical databases to ensure data integrity and accuracy. Perform preliminary data cleaning to remove irrelevant characters, spaces, line breaks, etc. to ensure the neatness of text data.

[0060] Step 2: Extract the electronic medical record text, associate it with the ICD code of the doctor's diagnosis, and build a training dataset.

[0061] For each patient's electronic medical record, the discharge summary section was selected as the electronic medical record text. The extracted electronic medical record text was associated with the ICD code of the doctor's diagnosis to form training data. Each patient's electronic medical record text may correspond to multiple ICD codes.

[0062] The electronic medical records of all patients and their corresponding ICD codes were collected to form an ICD automatic coding training dataset. All ICD codes that appeared in the training dataset were counted, and the ICD code of each training sample was summarized as the training set label.

[0063] Step 3: Build a multi-knowledge base and obtain auxiliary knowledge, that is, external knowledge obtained from multiple sources such as UMLS, unstructured knowledge bases, and large language models (LLMs), to bridge the semantic gap between clinical texts and diagnostic codes.

[0064] Build a UMLS synonym database: align ICD codes with UMLS concept unique identifiers (CUIs) to extract a collection of synonymous terms .

[0065] Build an unstructured knowledge base: retrieve ICD code-related entries, extract disease definitions, symptom descriptions and physiological characteristics, and build structured knowledge entries .

[0066] Using LLM to generate auxiliary knowledge base: design prompt templates (such as "Generate a detailed medical explanation of [ICD code], including symptoms, surgical process indicators and possible diagnostic methods"), and generate expanded text through the Qwen large model .

[0067] Step 4: Construct an ICD encoding model, which includes a context encoding module, an auxiliary knowledge screening module, a multiple attention interaction module, and a fusion prediction module.

[0068] (4.1) Constructing the context encoding module of the model: Using the RoBERTa pre-trained model, encode the clinical text in blocks and generate context representations:

[0069] ;

[0070] in, is the word vector of the c-th text block, is the output representation, T is the number of words in the block, and d is the hidden layer dimension.

[0071] (4.2) Construct the auxiliary knowledge module of the model: Encode multi-source knowledge items into vectors:

[0072] .

[0073] The maximum diversity selection (MDP) algorithm is used to select the M most unique pieces of knowledge from three knowledge sources: the UMLS (Unified Medical Language System), an unstructured knowledge base, and LLMs (Large Language Models). The optimization goal is to select the M knowledge vectors with the broadest semantic coverage. The specific problem is as follows:

[0074] ;

[0075] in, is the cosine distance, To select variables, we finally get .

[0076] (4.3) Constructing a multi-attention interaction module: This part consists of a label attention mechanism and a cross attention mechanism.

[0077] Calculate label self-attention: The core goal of label self-attention (LSA) is to model the global dependency between ICD codes and solve the problem of sparse semantics of low-frequency labels in long-tail distribution. Its input is the contextual representation of clinical text. and label embedding L; through linear transformation Will Mapped to query vector Q, key vector K is directly used The value vector V is The original semantic representation of ; the attention weight α is generated by the nonlinear activation function tanh and softmax calculation:

[0078] ;

[0079] in, , is a trainable parameter; next, by applying α to the context representation , computes a label-specific contextual representation , while the document-level tag-specific representation It is generated by linearly transforming the output of each block and summing them up:

[0080] ;

[0081] in, is a learnable weight matrix and C is the total number of blocks.

[0082] Each label corresponds to an independent semantic vector, explicitly modeling the dependencies between labels. For example, for the low-frequency co-occurrence of "diabetic ketoacidosis" and "hyperglycemia," label self-attention can capture their potential association through the weight α, enhancing the representation of low-frequency labels.

[0083] Calculate label-context cross attention: Label-context cross attention (LCCA) aims to align the fine-grained semantics of ICD codes and clinical texts, and accurately locate key text fragments related to diagnostic codes. Its input is the label embedding L and the clinical text representation ; The query vector Q directly uses L to represent the semantic requirements of the label; the key K and value V are both from , representing the contextual features of the text; the interaction representation is calculated by scaling the dot product attention:

[0084] ;

[0085] in, Is a learnable weight matrix, C is the total number of blocks. Output In the dataset, each label vector aggregates the most semantically relevant text segments. For example, for the label "acute myocardial infarction," LCCA can focus on key descriptions in the text, such as "chest pain lasting >30 minutes" and "ST segment elevation," suppressing irrelevant noise and improving the model's sensitivity to core features.

[0086] Computational Knowledge-Context Cross Attention: Knowledge-Context Cross Attention (KCCA) fills the semantic gap between clinical text and diagnostic codes by introducing external multi-source knowledge (UMLS synonyms, unstructured knowledge entries, LLMs generated text). The input is the filtered knowledge representation , M is the number of knowledge items, and d represents the hidden layer dimension of the model; For clinical text representation, query Q is , indicating the semantic requirements of external knowledge; the key K is , the value V is the average pooling result of the knowledge vector , to aggregate global knowledge semantics, attention is calculated as follows:

[0087] ;

[0088] Among them, CA stands for cross attention mechanism, and the final output is Injecting external knowledge semantics into text representations. For example, when the text describes "blood sugar level > 300 mg / dL," KCCA can be combined with LLM to generate the knowledge that "blood sugar level > 130 mg / dL may lead to diabetes," helping the model infer the implicit diagnosis "E11.9 (type 2 diabetes)," significantly improving the encoding robustness of non-explicit descriptions.

[0089] (4.4) Constructing a Fusion Prediction Module: The core goal of the prediction module is to fuse the labels, text, and knowledge representations generated by the multimodal interaction module and generate the final ICD probability distribution through a lightweight convolutional network. Its design balances computational efficiency with semantic expression. First, the outputs of label self-attention (LSA), label-context cross attention (LCCA), and knowledge-context cross attention (KCCA) are superimposed and concatenated into a unified representation:

[0090] ;

[0091] Where C is the number of text blocks, and 3d is the concatenation dimension of the three attention outputs. Then, through two-level one-dimensional convolution and nonlinear activation function, the fused representation is mapped to the probability distribution of ICD encoding:

[0092] ;

[0093] The first convolution layer is used to extract local semantic patterns from the fused representation, the LeakyReLU activation function is used to alleviate the gradient vanishing problem, and the second convolution layer is used to map the intermediate features to the ICD label space to generate a multi-label probability distribution.

[0094] Step 5: Use the binary cross entropy loss function to optimize the ICD coding model, output the probability of each ICD code, and realize automatic prediction of the ICD code of the electronic medical record text.

[0095] The ICD auto-encoding task is considered as a multi-label classification task, and the binary cross entropy loss function is used to measure the difference between the model's predicted probability and the true label. Therefore, the binary cross entropy loss function is used to optimize the model. Its mathematical definition is as follows:

[0096] ;

[0097] in, : represents the true label of the j-th ICD code (1 for existence, 0 for non-existence), represents the model's predicted probability for the j-th ICD code; is the total number of ICD coding labels.

[0098] In order to ensure the stability of model training and the optimality of the final results, AdamW is used as the optimizer. Taking the training on MIMIC III-50 as an example, the learning rate is set to The number of warm-up steps is set to 2000 to avoid initial oscillations in training. An early stopping mechanism is also used, with the trigger condition being that training ends when the Micro-F1 evaluation index does not improve in the validation set for three consecutive rounds to avoid overfitting.

[0099] Three core metrics are used: precision, F1 score, and area under the receiver operating characteristic (ROC) curve (AUC). Precision measures the proportion of true positive examples among labels predicted as positive, reflecting the reliability of the model's diagnosis and avoiding the risk of misdiagnosis. The F1 score, as the harmonic mean of precision and recall, further balances the two. Micro-F1 focuses on the overall performance of high-frequency labels, while Macro-F1 treats all labels equally, with particular attention paid to the prediction of rare diseases in long-tail distributions. AUC, by calculating the ranking ability of positive and negative sample prediction probabilities, provides a global assessment independent of thresholds, effectively measuring the model's robustness in scenarios with extreme class imbalance. These metrics collectively characterize the model's multi-dimensional performance, from accuracy, coverage, to generalization, providing a rigorous quantitative basis for clinical decision-making.

[0100] Table 1 Performance of each model on the MIMIC III-50 test set

[0101]

[0102] Table 2 Performance of each model on the MIMIC III-Full test set

[0103]

[0104] As shown in Table 1 and Table 2, the present invention achieved better results than other models on the two datasets of intensive care medical information database MIMIC III-50 and MIMIC-Full.

Claims

1. A method for automatically encoding the International Classification of Diseases, characterized in that: The following steps are involved: (1) Obtain electronic medical record data from the medical database and perform preliminary data cleaning; (2) Extract electronic medical record text, associate it with the ICD code of the doctor's diagnosis, and construct a training dataset; (3) Constructing a multi-knowledge base, including the UMLS synonym database, unstructured knowledge base, and auxiliary knowledge base generated by LLM; (4) Construct an ICD encoding model, including a context encoding module, an auxiliary knowledge screening module, a multiple attention interaction module, and a fusion prediction module; (5) Using the binary cross entropy loss function to optimize the ICD coding model, the probability of each ICD code is output, and the ICD code of the electronic medical record text is automatically predicted; The multi-attention interaction module in step (4) is composed of a label attention mechanism and a cross attention mechanism; Label self-attention is to model the global dependency between ICD codes and solve the problem of semantic sparsity of low-frequency labels in long-tail distribution; Input is a contextual representation of clinical text and label embedding L; through linear transformation Will Mapped to query vector Q, key vector K is directly used The value vector V is The original semantic representation of ; the attention weight α is generated by the nonlinear activation function tanh and softmax calculation: ； in, , is a trainable parameter; next, by applying α to the context representation , computes a label-specific contextual representation , while the document-level tag-specific representation It is generated by linearly transforming the output of each block and summing them up: ； in, is a learnable weight matrix, C is the total number of blocks; The cross-attention mechanism includes label-context cross-attention and knowledge-context cross-attention. Label-context cross-attention aims to align the fine-grained semantics of ICD codes and clinical texts, accurately locating key text fragments related to diagnostic codes; knowledge-context cross-attention fills the semantic gap between clinical texts and diagnostic codes by introducing external multi-source knowledge. The label-context cross attention input is the label embedding L and the clinical text representation ; The query vector Q directly uses L to represent the semantic requirements of the label; the key K and value V are both from , representing the contextual features of the text; the interaction representation is calculated by scaling the dot product attention: ； in, is a learnable weight matrix, C is the total number of blocks; output In , each label vector aggregates the text segments that are most semantically relevant to it; The knowledge-context cross attention introduces external multi-source knowledge input as the filtered knowledge representation , M is the number of knowledge items, and d represents the hidden layer dimension of the model; For clinical text representation, query Q is , indicating the semantic requirements of external knowledge; the key K is , the value V is the average pooling result of the knowledge vector , to aggregate global knowledge semantics, attention is calculated as follows: ； Among them, CA stands for cross attention mechanism, and the final output is Injecting external knowledge semantics into textual representations.

2. The method for automatically encoding the International Classification of Diseases according to claim 1, wherein: The implementation process of step (2) is as follows: For each patient's electronic medical record, the discharge summary section is selected as the electronic medical record text. The extracted electronic medical record text is associated with the ICD code of the doctor's diagnosis to form training data. The electronic medical record texts of all patients and their corresponding ICD codes are summarized to form the ICD automatic coding training dataset. All ICD codes appearing in the training dataset are counted, and the ICD code of each training sample is summarized as the training set label.

3. The method for automatically encoding the International Classification of Diseases according to claim 1, wherein: The implementation process of step (3) is as follows: UMLS Thesaurus: Extract synonymous terms by aligning ICD codes with UMLS Concept Unique Identifiers (CUIs) ; Unstructured knowledge base: retrieve ICD code related entries, extract disease definitions, symptom descriptions and physiological characteristics, and construct structured knowledge entries ; Using LLM to generate auxiliary knowledge base: design prompt templates and generate expanded text through Qwen large model .

4. The method for automatically encoding the International Classification of Diseases according to claim 1, wherein: The context encoding module in step (4) is: Using the RoBERTa pre-trained model, we encode clinical text in blocks and generate contextual representations: ； in, is the word vector of the c-th text block, is the output representation, T is the number of words in the block, and d is the hidden layer dimension.

5. The method for automatically encoding the International Classification of Diseases according to claim 1, wherein: The auxiliary knowledge screening module in step (4) is: Encode multi-source knowledge items into vectors: ； Using the maximum diversity screening algorithm MDP, we select M knowledge vectors with the widest semantic coverage. The specific problems are as follows: ； in, is the cosine distance, To select variables, we finally get .

6. The method for automatically encoding the International Classification of Diseases according to claim 1, wherein: The fusion prediction module fuses the labels, texts, and knowledge representations generated by the multimodal interaction module and generates the final ICD probability distribution through a lightweight convolutional network. The outputs of label self-attention, label-context cross-attention, and knowledge-context cross-attention are superimposed and spliced ​​into a unified representation: ； Where C is the number of text blocks, and 3d is the concatenation dimension of the three attention outputs. Then, through two-level one-dimensional convolution and nonlinear activation function, the fused representation is mapped to the probability distribution of ICD encoding: ； Among them, the first layer of convolution is used to extract local semantic patterns from the fusion representation, the LeakyReLU activation function is used to alleviate the gradient vanishing problem, and the second layer of convolution is used to map the intermediate features to the ICD label space to generate a multi-label probability distribution.

7. The method for automatically encoding the International Classification of Diseases according to claim 1, wherein: The implementation process of optimizing the ICD coding model using the binary cross entropy loss function in step (5) is as follows: ； in, , represents the true label of the j-th ICD code, 1 means existence, 0 means non-existence, represents the model's predicted probability for the j-th ICD code; is the total number of ICD code labels.

Images