A hierarchical ICD coding method and system combining coarse and fine granularity

Abstract

The present invention discloses a hierarchical ICD encoding method and system combining coarse and fine granularity, comprising the following steps: S1: inputting an electronic medical record text and a corresponding standard disease name; S2: obtaining a doctor's diagnosis in the electronic medical record text, and then preprocessing the electronic medical record text, the doctor's diagnosis and the standard disease name; S3: coarse-grained screening: constructing a first word vector for the preprocessed doctor's diagnosis, and performing a first encoding and a first decoding on the first word vector to obtain the first three digits of the ICD code, and obtaining the corresponding ICD code and the standard disease name as candidate items; S4: fine-grained screening: converting the preprocessed electronic medical record text and the standard disease name in the candidate item in S3 into a second word vector sequence and performing a second encoding, and then performing a similarity score on the second encoding result; S5: taking the ICD code with the highest similarity score as the output result.

Description

Technical Field

[0001] The present invention relates to the field of ICD coding technology, and in particular to a hierarchical ICD coding method and system combining coarse and fine granularity. Background Art

[0002] The International Classification of Diseases (ICD) is a coding system that categorizes diseases based on certain characteristics and represents them using a code. Disease codes facilitate the identification of diagnoses and procedures during a patient's visit. These codes are used in clinical research and healthcare, such as epidemiological studies and billing.

[0003] Manual ICD coding requires coders to read and annotate complex and lengthy electronic medical records, resulting in many errors. Consequently, a growing number of studies have focused on using machine learning methods for automated ICD coding. Automated ICD coding is difficult for two reasons: Electronic medical record text is very long and contains a large amount of useless noise, making feature extraction difficult. For example, in the MIMIC III dataset, the average text length exceeds 1,500 words, and the vast majority of these words contain useless information. ICD codes are numerous and suffer from a serious long-tail problem, with a large number of common diseases dominating medical text, while the vast majority of rare diseases rarely appear in the dataset.

[0004] Existing technical solutions can be roughly divided into two types according to the input corpus: (1): Treating ICD coding as a multi-label classification problem, using electronic medical record text as input corpus, using traditional machine learning methods or deep learning methods, and training N binary classifiers to obtain results. This includes traditional machine learning methods (such as SVM, LR, etc.) and deep learning methods, such as RNN and CNN. CNN-based deep learning models use single or multiple fixed-size convolution filters to capture text fragments related to ICD codes in electronic medical record text, but there are problems such as difficulty in selecting the optimal window. Another type of deep learning model is based on RNN to handle the dependencies of electronic medical record text, but due to limited information transmission capacity and gradient disappearance, the expression ability of RNN is limited for long documents. Due to the serious long-tail distribution of diseases in electronic medical records, the classifier for rare diseases has poor generalization ability due to insufficient training samples. (2): Using doctor's diagnosis as input corpus, treating the ICD coding problem as a multi-classification problem or sequence generation problem, using traditional machine learning or deep learning methods to construct multiple classifiers or encoder-decoders. Some work uses traditional machine learning classifiers, such as the K-nearest neighbor algorithm and Bayesian independent classifiers, to construct N classifiers. However, due to the long-tail distribution, the accuracy for rare diseases is low. Some work considers this task a sequence generation problem, generating ICD codes from physician diagnoses. Sequence generation performs well at coarse-grained levels, but errors increase with finer granularity. Furthermore, the information in physician diagnoses is less complete than in electronic medical records. The use of abbreviations and synonyms in physician diagnoses can also impact models using a single physician's diagnosis. Summary of the Invention

[0005] In response to the problems of insufficient utilization of hierarchical information in ICD coding in the existing technology and the easy occurrence of misclassification in fine-grained ICD coding, resulting in low ICD coding accuracy, the present invention proposes a hierarchical ICD coding method and system that combines coarse and fine granularity. Hierarchical ICD coding is performed by using two corpora, electronic medical record text and doctor diagnosis, to improve ICD coding accuracy.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] 1 A hierarchical ICD coding method combining coarse and fine granularity, comprising the following steps:

[0008] S1: Input the electronic medical record text and the corresponding standard disease name;

[0009] S2: Obtain the doctor's diagnosis in the electronic medical record text, and then preprocess the electronic medical record text, doctor's diagnosis and standard disease name;

[0010] S3: Coarse-grained screening: Construct the first word vector for the preprocessed doctor's diagnosis, perform the first encoding and first decoding on the first word vector, obtain the first three digits of the ICD code, and obtain the corresponding ICD code and standard disease name as candidate items;

[0011] S4: Fine-grained screening: convert the pre-processed electronic medical record text and the standard disease names in the S3 output candidate items into corresponding second word vector sequences and perform second encoding, and then perform similarity scoring on the second encoding results;

[0012] S5: The ICD code with the highest similarity score is used as the output result.

[0013] Preferably, in said S2, the preprocessing step is: using the Jieba word segmentation script to segment the electronic medical record text, doctor's diagnosis and standard disease name, remove stop words and remove useless noise.

[0014] Preferably, the S3 includes:

[0015] S3-1: Construct the first word vector, converting the preprocessed doctor's diagnosis into a word vector sequence E = e d1 ,e d2 ,…,e dn ], e dn Represents the nth first word vector of the doctor's diagnosis text d;

[0016] S3-2: Input the word vector sequence E into the first encoder for encoding to obtain a feature vector;

[0017] S3-3: Input the feature vector output by the encoder into the first decoder for decoding, and output the first three bits of the ICD code;

[0018] S3-4: After obtaining the first three digits of the ICD code, all corresponding ICD codes and corresponding standard disease names are extracted as candidates.

[0019] Preferably, in S3-2, the first encoder is composed of 6 identical encoding layers, each encoding layer includes two sublayers: a self-attention layer and a position-aware feedforward network layer.

[0020] Preferably, in S3-3, the first decoder includes a self-attention layer, a position-aware feedforward network layer and a third sublayer, and the third sublayer performs multi-head attention on the output of the first encoder and outputs the first three bits of the ICD code.

[0021] Preferably, the decoder is updated using a cross entropy loss function as follows:

[0022]

[0023] In formula (1), LOSS represents the cross entropy loss function; T represents the total number of decoding steps; Represents the probability of correctly predicting the corresponding word.

[0024] Preferably, the S4 includes:

[0025] S4-1: Construct the second word vector, convert the preprocessed electronic medical record text and the disease standard name in the candidate item in S3 into the corresponding second word vector sequence and represents the second word vector of the mth electronic medical record, and w represents the electronic medical record; represents the second word vector of the nth disease standard name, and icd represents the disease standard name;

[0026] S4-2: Encode the second word vector sequence e using the second encoder and output the encoding result;

[0027] S4-3: Use a multi-hop attention mechanism network to obtain feature vectors of electronic medical record text and standard disease names from the encoding results;

[0028] S4-4: Perform similarity scoring on the feature vectors of electronic medical record text and standard disease names.

[0029] Preferably, in S4-2, the encoding process of the second encoder is:

[0030]

[0031]

[0032]

[0033]

[0034]

[0035]

[0036] In formula (2), is the output of the forward propagation LSTM of the electronic medical record text; represents LSTM forward propagation; Represents the word vector sequence corresponding to the electronic medical record text; Represents the LSTM output of the electronic medical record text backpropagation; represents LSTM backward propagation; Represents the output of the forward propagation LSTM of the disease standard name; Represents the output of the back-propagation LSTM of the disease standard name; represents the encoding result of the electronic medical record text in step i; ⊕ represents the addition operation of the vector; Indicates the encoding result of the i-th step of the standard disease name.

[0037] Preferably, in S4-3, the calculation formula for the similarity score is:

[0038]

[0039]

[0040] In formula (3), sim(r,icd) represents the similarity score of the feature vectors of the electronic medical record text and the disease standard name; r represents the electronic medical record text; icd represents the disease standard name; K represents the number of hops in the hopping attention network; Represents the feature vector output of the electronic medical record text after passing through the multi-hop attention network; Represents the feature vector output by the multi-head self-attention network of the standard disease name;

[0041] Loss represents the loss function; M represents the hyperparameter used to separate the correct ICD code from the incorrect code; icd + The feature vector output of the disease standard name corresponding to the correct ICD code in the electronic medical record text after passing through the multi-hop attention network; icd - The feature vector output of the standard disease name corresponding to the incorrect ICD code in the electronic medical record text after passing through the multi-hop attention network.

[0042] The present invention also provides a hierarchical ICD coding system combining coarse and fine granularity, comprising:

[0043] A data preprocessing module is used to obtain the doctor's diagnosis from the electronic medical record text and preprocess the electronic medical record text, doctor's diagnosis and standard disease name;

[0044] The ICD code coarse classification module is used to construct the first word vector sequence of the preprocessed doctor's diagnosis, and then encode and decode it to obtain the first three digits of the ICD code. The ICD code and the corresponding standard disease name belonging to the general category are then extracted as candidates;

[0045] The ICD coding segmentation module is used to construct a second word vector sequence for the preprocessed electronic medical record text and disease standard name, and then input the second word vector sequence into the multi-hop attention network for similarity scoring, and select the one with the highest score as the ICD coding result.

[0046] In summary, due to the adoption of the above technical solution, compared with the prior art, the present invention has at least the following beneficial effects:

[0047] The present invention uses physician diagnoses with rich information as input text for coarse classification to perform coarse-grained classification of ICD codes, which can achieve a high accuracy rate and exclude most completely dissimilar ICD codes, thus alleviating the long-tail problem to a certain extent.

[0048] Then, a multi-hop attention network is used to focus on different parts of similar diseases under the same category and strengthen their weights, which can obtain higher scores in the final multiple similarity scores, realize the precise matching mechanism of ICD fine-grained coding, and further improve the accuracy of ICD automatic coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of a hierarchical ICD coding method combining coarse and fine granularity according to an exemplary embodiment of the present invention.

[0050] Figure 2 Schematic diagram of a hierarchical ICD coding system combining coarse and fine granularity according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below with reference to the examples and specific implementation methods. However, this should not be understood as limiting the scope of the present invention to the following examples, as all technologies implemented based on the present invention fall within the scope of the present invention.

[0052] In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0053] like Figure 1 As shown, the present invention provides a hierarchical ICD coding method combining coarse and fine granularity, which specifically includes the following steps:

[0054] S1: Input the electronic medical record text and the corresponding standard disease name;

[0055] S2: Obtain the doctor's diagnosis in the electronic medical record text, and then preprocess the electronic medical record text, doctor's diagnosis and standard disease name.

[0056] In this embodiment, regular expressions are used to segment the doctor's diagnosis in the medical record text.

[0057] The preprocessing steps are: using the jieba word segmentation script to segment the electronic medical record text, doctor's diagnosis and standard disease name, remove stop words and remove useless noise.

[0058] S3: Coarse-grained screening: Construct the first word vector for the preprocessed doctor's diagnosis, and perform the first encoding and first decoding on the first word vector to obtain the first three digits of the ICD code. All ICD codes and standard disease names belonging to the first three digits of the ICD code are selected as candidates.

[0059] S3-1: Construct the first word vector. Word Embedding uses the GloVe vector pre-trained by the dataset to vectorize the word. The pre-processed doctor diagnosis W = [w d1 ,w d2 ,…,w dn ] is converted into a word vector sequence E = [e d1 ,e d2 ,…,e dn ],w dn represents the nth word in the doctor's diagnosis text d; e dn Represents the nth first word vector of the doctor's diagnosis text d; the first word vector does not participate in the training of the decoder.

[0060] S3-2: Input the word vector sequence E into the first encoder (Transformer encoder network) for encoding to obtain a feature vector.

[0061] In this embodiment, the first encoder is composed of 6 identical encoding layers, each of which includes two sublayers: a self-attention layer and a position-aware feedforward network layer.

[0062] S3-3: Input the feature vector output by the encoder into the first decoder for decoding, and output the first three bits of the ICD code.

[0063] In this embodiment, the first decoder has a similar structure to the first encoder. In addition to the self-attention layer and the position-aware feedforward network layer, the first decoder also inserts a third sublayer, which performs multi-head attention on the output of the first encoder stack and outputs the first three bits of the ICD code.

[0064] In this embodiment, the cross entropy loss function is used to update the decoder, and the update method is as follows:

[0065]

[0066] In formula (1), LOSS represents the cross entropy loss function; T represents the total number of decoding steps; Represents the probability of correctly predicting the corresponding word.

[0067] S3-4: After obtaining the first three digits of the ICD code, all ICD codes and corresponding disease standard names belonging to the major categories corresponding to the first three digits of the ICD code are extracted as candidates.

[0068] S4: Fine-grained screening: The preprocessed electronic medical record text and the standard disease names in the candidate items in S3 are converted into a second word vector sequence and encoded in the second way, and then the similarity of the second encoding results is scored.

[0069] S4-1: Construct the second word vector. Word Embedding uses the GloVe vector pre-trained by the dataset for word vectorization. The pre-processed electronic medical record text and the disease standard name in the candidate in S3 are converted into the second word vector sequence respectively. and represents the second word vector of the mth electronic medical record, and w represents the electronic medical record; represents the second word vector of the nth disease standard name, and icd represents the disease standard name;

[0070] S4-2: Encode the second word vector sequence e using the second encoder (Bi-LSTM). The encoding process is as follows:

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] In formula (2), is the output of the forward propagation LSTM of the electronic medical record text; Represents LSTM

[0078] Forward propagation; Represents the word vector corresponding to the electronic medical record text; Represents the LSTM output of the electronic medical record text backpropagation; represents LSTM backward propagation; Represents the output of the forward propagation LSTM of the disease standard name; Represents the output of the back-propagation LSTM of the disease standard name; represents the encoding result of the i-th electronic medical record text; ⊕ represents the addition operation of the vector; Represents the coding result of the standard name of each disease i.

[0079] S4-3: Use the MultiHeadAttention network to obtain the feature vectors of the electronic medical record text and the standard disease name. The specific calculation is as follows:

[0080]

[0081]

[0082] In formula (3), The feature vector representing the standard name of the disease; MultiHeadAttention represents the multi-hop attention mechanism; H icd A set of coding results representing the standard names of diseases; Indicates the middle value; represents the network parameters; h r(t) Represents the hidden vector of the t-th step of the electronic medical record text; represents the network parameters, m represents the memory vector; k represents the number of hops; is a separate memory vector; r represents the electronic medical record text; Represents the obtained weight parameter; Represents network parameters; express The transpose of Feature vector representing the electronic medical record text.

[0083] Is a separate memory vector used to guide the next attention step, recursive method and initial value as follows:

[0084]

[0085]

[0086] In formula (4), represents the memory vector representing the electronic medical record text; Feature vector representing the electronic medical record text; represents the initial value of the feature vector representing the electronic medical record text; l represents the length of the electronic medical record.

[0087] S4-4: Use cosine similarity sim to score the similarity between the feature vectors of the electronic medical record text and the standard disease name. The loss function uses the hinge loss function:

[0088]

[0089] Loss=max{0,M-sim(r,icd + )+sim(r,icd - )} (5)

[0090] In formula (5), sim(r,icd) represents the similarity score of the feature vectors of the electronic medical record text and the disease standard name; r represents the electronic medical record text; icd represents the disease standard name; K represents the number of hops in the hopping attention network; Represents the feature vector output of the electronic medical record text after passing through the multi-hop attention network; Represents the feature vector output by the multi-head self-attention network of the standard disease name.

[0091] Loss represents the loss function; M represents the hyperparameter used to separate the correct ICD code from the incorrect code; icd + The feature vector output of the disease standard name corresponding to the correct ICD code in the electronic medical record text after passing through the multi-hop attention network; icd - The feature vector output of the standard disease name corresponding to the incorrect ICD code in the electronic medical record text after passing through the multi-hop attention network.

[0092] S5: The ICD code with the highest similarity score is used as the output result.

[0093] The present invention uses doctor diagnoses with rich information as input text for coarse classification to perform coarse-grained classification of ICD codes, which can achieve a high accuracy rate, while also excluding most completely dissimilar ICD codes and alleviating the long-tail problem to a certain extent.

[0094] A multi-hop attention network is then used to implement a precise matching mechanism for ICD fine-grained coding. Compared to previous models, this system can focus on different parts of similar diseases within the same general category and strengthen their weights, achieving higher scores in the final multiple similarity scoring, further improving the accuracy of ICD automatic coding. During the segmentation stage, taking "Hypertension I," "Hypertension II," and "Hypertension III" as examples, which all belong to I10, the multi-hop attention network can focus on more information about "I," "II," and "III," effectively distinguishing similar diseases and reducing the problem of misclassification of similar ICD codes at the fine-grained level.

[0095] Based on the above method, Figure 2 As shown, the present invention also provides a hierarchical ICD coding system combining coarse and fine granularity, including a data preprocessing module, an ICD coding coarse segmentation module and an ICD coding subdivision module; the output end of the data preprocessing module is connected to the input end of the ICD coding coarse segmentation module, the output end of the ICD coding coarse segmentation module is connected to the input end of the ICD coding subdivision module, and the ICD coding subdivision module outputs the ICD coding result.

[0096] A data preprocessing module is used to obtain the doctor's diagnosis from the electronic medical record text and preprocess the electronic medical record text, doctor's diagnosis and standard disease name;

[0097] In this embodiment, regular expressions are used to segment the doctor's diagnosis in the medical record text.

[0098] The preprocessing steps are: using the jieba word segmentation script to segment the electronic medical record text, doctor's diagnosis and standard disease name, remove stop words and remove useless noise.

[0099] The ICD code coarse classification module is used to construct the first word vector sequence of the preprocessed doctor's diagnosis, and to encode and decode it to obtain the first three digits of the ICD code. The ICD codes and corresponding standard disease names belonging to this major category are then extracted as candidates.

[0100] The ICD coding segmentation module is used to construct a second word vector sequence for the preprocessed electronic medical record text and disease standard name, and then input the second word vector sequence into the multi-hop attention network for similarity scoring, and select the one with the highest score as the ICD coding result.

[0101] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present invention, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A hierarchical ICD coding method combining coarse and fine granularity, characterized in that: The following steps are involved: S1: Input the electronic medical record text and the corresponding standard disease name; S2: Obtain the doctor's diagnosis in the electronic medical record text, and then preprocess the electronic medical record text, doctor's diagnosis and standard disease name; S3: Coarse-grained screening: Construct the first word vector for the preprocessed doctor's diagnosis, perform the first encoding and first decoding on the first word vector, obtain the first three digits of the ICD code, and obtain the corresponding ICD code and standard disease name as candidate items; S4: Fine-grained screening: convert the pre-processed electronic medical record text and the standard disease names in the S3 output candidate items into corresponding second word vector sequences and perform second encoding, and then perform similarity scoring on the second encoding results; The S4 includes: S4-1: Construct the second word vector, convert the preprocessed electronic medical record text and the disease standard name in the candidate item in S3 into the corresponding second word vector sequence and represents the second word vector of the mth electronic medical record, and w represents the electronic medical record; represents the second word vector of the nth disease standard name, and icd represents the disease standard name; S4-2: Encode the second word vector sequence e using the second encoder and output the encoding result; Among them, the encoding process of the second encoder is: In formula (2), is the output of the forward propagation LSTM of the electronic medical record text; represents LSTM forward propagation; Represents the word vector sequence corresponding to the electronic medical record text; Represents the LSTM output of the electronic medical record text backpropagation; represents LSTM backward propagation; Represents the output of the forward propagation LSTM of the disease standard name; Represents the output of the back-propagation LSTM of the disease standard name; Indicates the coding result of step i of the electronic medical record text; Represents the addition operation of a vector; Indicates the coding result of the standard disease name in step i; S4-3: Use a multi-hop attention mechanism network to obtain feature vectors of electronic medical record text and standard disease names from the encoding results; S4-4: Similarity scoring between feature vectors of electronic medical record text and standard disease names; S5: The ICD code with the highest similarity score is used as the output result.

2. The hierarchical ICD coding method combining coarse and fine granularity according to claim 1, characterized in that: In S2, the preprocessing steps are: using the jieba word segmentation script to segment the electronic medical record text, doctor's diagnosis and standard disease name, remove stop words and remove useless noise.

3. The hierarchical ICD coding method combining coarse and fine granularity as claimed in claim 1, characterized in that: The S3 includes: S3-1: Construct the first word vector, converting the preprocessed doctor's diagnosis into a word vector sequence E = e d1 ,e d2 ,…,e dn ], e dn Represents the nth first word vector of the doctor's diagnosis text d; S3-2: Input the word vector sequence E into the first encoder for encoding to obtain a feature vector; S3-3: Input the feature vector output by the encoder into the first decoder for decoding, and output the first three bits of the ICD code; S3-4: After obtaining the first three digits of the ICD code, all corresponding ICD codes and corresponding standard disease names are extracted as candidates.

4. The hierarchical ICD coding method combining coarse and fine granularity as claimed in claim 3, characterized in that: In S3-2, the first encoder is composed of 6 identical encoding layers, each of which includes two sublayers: a self-attention layer and a position-aware feedforward network layer.

5. The hierarchical ICD coding method combining coarse and fine granularity as claimed in claim 3, characterized in that: In S3-3, the first decoder includes a self-attention layer, a position-aware feedforward network layer, and a third sublayer. The third sublayer performs multi-head attention on the output of the first encoder and outputs the first three bits of the ICD code.

6. The hierarchical ICD coding method combining coarse and fine granularity as claimed in claim 5, characterized in that: The decoder is updated using the cross entropy loss function as follows: In formula (1), LOSS represents the cross entropy loss function; T represents the total number of decoding steps; Represents the probability of correctly predicting the corresponding word.

7. The hierarchical ICD coding method combining coarse and fine granularity as claimed in claim 1, characterized in that: In S4-3, the calculation formula for the similarity score is: Loss=max{0,M-sim(r,icd + )+sim(r,icd - )} (3) In formula (3), sim(r,icd) represents the similarity score of the feature vectors of the electronic medical record text and the disease standard name; r represents the electronic medical record text; icd represents the disease standard name; K represents the number of hops in the hopping attention network; Represents the feature vector output of the electronic medical record text after passing through the multi-hop attention network; Represents the feature vector output by the multi-head self-attention network of the standard disease name; Loss represents the loss function; M represents the hyperparameter used to separate the correct ICD code from the incorrect code; icd + The feature vector output of the disease standard name corresponding to the correct ICD code in the electronic medical record text after passing through the multi-hop attention network; icd - The feature vector output of the standard disease name corresponding to the incorrect ICD code in the electronic medical record text after passing through the multi-hop attention network.

8. A hierarchical ICD coding system combining coarse and fine granularity based on the method of any one of claims 1 to 7, characterized in that: include: A data preprocessing module is used to obtain the doctor's diagnosis from the electronic medical record text and preprocess the electronic medical record text, doctor's diagnosis and standard disease name; The ICD code coarse classification module is used to construct the first word vector sequence of the preprocessed doctor's diagnosis, and then encode and decode it to obtain the first three digits of the ICD code. Then, the ICD codes and corresponding standard disease names belonging to the major categories corresponding to the first three digits of the ICD code are extracted as candidates; The ICD coding segmentation module is used to construct a second word vector sequence for the preprocessed electronic medical record text and disease standard name, and then input the second word vector sequence into the multi-hop attention network for similarity scoring, and select the one with the highest score as the ICD coding result.

Images