An ICD disease classification-oriented few-shot learning method

Abstract

A few-shot learning method for ICD disease classification, using an automatic ICD coding method based on deep learning, learning the standard name of each code in the international disease classification using a gating mechanism and regularization method, the standard name of each code in the international disease classification as an additional knowledge, and through a gating mechanism and regularization method, the standard name of the ICD code is integrated into the model, which can enrich the features learned by the model, so that the model can learn a small amount of samples to achieve high classification performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical treatment, in particular to an ICD disease classification-oriented few-shot learning method. BACKGROUND

[0002] International Statistical Classification of Diseases and Related Health Problems, referred to as International Classification of Diseases or ICD code, is a classification system for human health status maintained by the World Health Organization. The ICD coding task is to map the health status (including diseases, symptoms, signs, surgeries and treatments, etc.) of patients in electronic medical records into standard ICD codes composed of numbers and letters, which is a classification problem. ICD coding can standardize different forms of clinical records and plays an important role in hospital information systems, clinical decision-making, epidemiological analysis and medical insurance, etc.

[0003] The ICD coding task needs to be completed by a system-trained coder, who needs to read the text content in the electronic medical record throughout the article and assign appropriate ICD codes according to the diagnosis and treatment of the patient recorded therein. Manual coding is a low-efficiency work. The existing automatic ICD coding method is a multi-label classification method based on supervised learning. This kind of method needs a large amount of manually annotated high-quality data for model training, and needs to mark the corresponding ICD code for each medical record text. However, ICD coding has the characteristic of uneven sample size of each coding category, such as a large number of patients with "hypertension" and "diabetes", and a small number of patients with specific site malignant tumors (such as "multiple myeloma"). In this case, when the automatic coding model performs multi-label classification, the model is under-learned for the sample-poor category, resulting in poor classification performance of the model for the sample-poor category. SUMMARY

[0004] The present application provides a method that enables the model to learn less sample size and achieve high classification performance.

[0005] The technical scheme adopted by the present application to overcome the technical problems is:

[0006] An ICD disease classification-oriented few-shot learning method, comprising the following steps:

[0007] a) randomly initializing a word vector matrix Δ, is a real number space, V is the total number of words in the dictionary, and d is the dimension of the sub-vector.

[0008] b) Obtain the electronic medical record text. The character vector matrix Δ maps each character in the electronic medical record text to a vector through the index in the dictionary, resulting in the character vector representation E of the electronic medical record text, E = {e1, e2, ..., e...}. i ,...,e L}, where e i Let e ​​be the character vector corresponding to the i-th character in the electronic medical record text. i Let be a d-dimensional vector, i∈{1,...,L}, where L is the total number of characters in the electronic medical record text. c) Obtain the vector representation of the standard name of each ICD code in the word vector representation E by querying the word vector matrix Δ. The vector representation of the standard name of the m-th ICD code is c. m , Let be the word vector corresponding to the j-th character in the m-th ICD-coded standard name, j∈{1,...,|m|}, where |m| is the total number of characters in the m-th ICD-coded standard name, m∈{1,...,|C|}, and |C| is the number of ICD-coded categories in E represented by the word vector. Let c be the vector representation of the standard name of the m-th ICD code. m Max pooling is performed along the |m| dimension to obtain a vector representation of the standard name of the m-th ICD code.

[0009] d) Construct a set from the vector representations of the standard names of all max-pooled ICD codes obtained in step c).

[0010] e) Initialize a trainable parameter vector for each ICD code, resulting in U = {u1, u2, ..., u...} m ,...,u |C|}, where u m Let m be the trainable parameter vector of the m-th ICD encoding, where m∈{1,...,|C|}.

[0011] f) Calculate the gate weight Γ. Where γ m Let γ be the gating weight of the m-th ICD code, where m∈{1,...,|C|}. m It is a d-dimensional vector. The gating weight Γ is a floating-point number between 0 and 1, excluding 0 and 1;

[0012] g) Calculate the vector representation C of the ICD encoding;

[0013] h) Establish a set K of the occurrences of all codes in the data, K = {k1, k2, ..., k m ,...,k |C|}, where k m Let Γ be the number of times the m-th ICD code appears in the manually verified ICD code dataset, where m∈{1,...,|C|}, for Γ={γ1,γ2,...,γ m ,...,γ |C| Define a loss function for a constraint. In the formula, ||·||2 is the L2 norm. max(K) is the maximum value in set K, min(K) is the minimum value in set K, and τ is a hyperparameter;

[0014] i) By formula Calculate the probability of matching the currently input electronic medical record text with all |C| categories of ICD codes, and complete the establishment of the multi-label classification model. The value ranges from 0 to 1, excluding 0 and 1, when the probability The index corresponding to a value greater than 0.5 is m. Then m is the index of the multi-label classification model that assigns the current input electronic medical record text to the m-th ICD code. In the formula, σ(·) is the sigmoid function, T is the matrix transpose, and V=α T E, Z = EC T , ∑ L exp(Z) is calculated by summing along the Lth dimension of exp(Z);

[0015] j) By formula The cross-entropy loss function is calculated. In the formula, Y represents the manually verified accurate label corresponding to the input electronic medical record text. The value of label Y is either 0 or 1. If the index corresponding to the value 1 in label Y is m, then m represents the current input electronic medical record text and the m-th ICD code that match, as shown by the formula. The overall loss was calculated. In the formula, λ is a hyperparameter, and the overall loss is calculated using gradient descent. Minimize the optimization of the multi-label classification model.

[0016] Furthermore, in step a), the values ​​in the word vector matrix Δ are initialized to... The random number obtained from the middle This represents a normal distribution with a mean of 0 and a standard deviation of 1.

[0017] Further, the vector representation of the mth ICD code after max-pooling is calculated by formula

[0018] Further, the values in the set U in step e) are initialized as random numbers obtained from

[0019] Further, the gating weight Γ is calculated by formula Γ=σ(QW2) in step f), where σ(·) is the sigmoid function, W2 is a trainable parameter matrix, where W1 is a trainable parameter matrix, || is the matrix concatenation in the dimension of d, The dimension of is |C|×2d.

[0020] Further, the vector representation C of the ICD code is calculated by formula

[0021] Preferably, τ=1000 in step h).

[0022] Preferably, λ=0.0001 in step j).

[0023] The present application has the beneficial effect that: by using the automatic ICD coding method based on deep learning, the standard name of each code in the international disease classification (such as the ICD code and the standard name of the ICD code shown in Table 1) is learned as a kind of additional knowledge by using a gating mechanism and a regularization method, and the standard name of the ICD code is integrated into the model by using a gating mechanism and a regularization method, which can enrich the features learned by the model, so that the model can achieve high classification performance with less sample size. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The method flowchart of the present application is shown in Figure 1.

[0025] Figure 2 The influence curve of different τ values on ||γ m ||2 is shown in Figure 2.

[0026] Figure 3 The influence curve of different τ values on k′ m is shown in Figure 3. DETAILED DESCRIPTION

[0027] The present application will be described in detail below with reference to the accompanying drawings. Figure 1 , and Figure 3.​​​Figure 2 , attached Figure 3 The application is further described.

[0028] In view of the limitations of the prior art, a few-sample learning method for automatic ICD coding is proposed, which is an automatic ICD coding method based on deep learning, uses a gating mechanism and a regularization method to learn the standard name of each code in the international disease classification (such as the ICD code and the standard name of the ICD code shown in Table 1), and uses the standard name of each code in the international disease classification as additional knowledge, and through a gating mechanism and a regularization method, the standard name of the ICD code is integrated into the model, which can enrich the features learned by the model, so that the model can learn a small amount of samples to achieve high classification performance.

[0029]

[0030] Table 1 ICD code and standard name of ICD code

[0031] The model proposed in the application is a multi-label classification model using electronic medical record text, that is, inputting an electronic medical record text, the model assigns several appropriate ICD codes according to the electronic medical record text, and a patient may have multiple diseases, so an electronic medical record text may correspond to multiple ICD codes, so it is a multi-label classification (Multi-label Classification) problem.

[0032] Specifically, as shown in the attached Figure 1 A few-sample learning method for ICD disease classification includes the following steps:

[0033] a) Randomly initialize a word vector matrix Δ, and the dimension of the word vector matrix Δ is is a real number space, V is the total number of words in the dictionary, and d is the dimension of the sub-vector. The word vector matrix can map each word in the text to a vector through the index in the dictionary, and the word vector matrix is a trainable model parameter.

[0034] b) Obtain an electronic medical record text, and the word vector matrix Δ maps each word in the electronic medical record text to a vector through the index in the dictionary to obtain the word vector representation E of the electronic medical record text, E={e1,e2,...,e i ,...,e L}, wherein e i is the word vector corresponding to the i-th word in the electronic medical record text, e i is a d-dimensional vector, i∈{1,...,L}, L is the total number of words in the electronic medical record text, and the dimension of the word vector representation E of the electronic medical record text is

[0035] c) Obtain the vector representation of the standard name of each ICD code in the vector representation E by querying the word vector matrix Δ, and the vector representation of the standard name of the mth ICD code is c m , is the word vector corresponding to the jth word in the standard name of the mth ICD code, j ∈ {1,..., |m|}, |m| is the total number of words in the standard name of the mth ICD code, m ∈ {1,..., |C|}, |C| is the number of categories of ICD codes in the vector representation E, and c m has a dimension of The vector representation c m of the standard name of the mth ICD code is obtained by performing max-pooling on the |m| dimensions, and the vector representation of the standard name of the mth ICD code after max-pooling is has a dimension of

[0036] d) Construct a set of all vector representations of ICD code standard names after max-pooling obtained in step c) The set has a dimension of

[0037] e) Initialize a trainable parameter vector for each ICD code to obtain U = {u1, u2,..., u m ,...,u |C|}, where u m is the trainable parameter vector of the mth ICD code, m ∈ {1,..., |C|}, and u m has a dimension of The dimension of U is

[0038] f) Calculate the gating weight Γ, where γ m is the gating weight of the mth ICD code, m ∈ {1,..., |C|}, and γ m is a d-dimensional vector, and the dimension of the gating weight Γ is The gating weight Γ takes a floating-point number between 0 and 1 and does not include 0 and 1.

[0039] g) Calculate the vector representation C of the ICD code.

[0040] h) Establish a set K of the number of times each code appears in the data, K = {k1, k2,..., k m ,...,k |C|}, where k mLet Γ be the number of times the m-th ICD code appears in the manually verified ICD code dataset, where m∈{1,...,|C|}, for Γ={γ1,γ2,...,γ m ,...,γ |C| Define a loss function for a constraint. In the formula, ||·||2 is the L2 norm. max(K) is the maximum value in set K, min(K) is the minimum value in set K, and τ is a hyperparameter. Preferably, in a specific embodiment of the present invention, the model performs best when τ = 1000.

[0041] i) By formula Calculate the probability of matching the currently input electronic medical record text with all |C| categories of ICD codes, and complete the establishment of the multi-label classification model. The value ranges from 0 to 1, excluding 0 and 1, when the probability The index corresponding to a value greater than 0.5 is m. Then m is the index of the multi-label classification model that assigns the current input electronic medical record text to the m-th ICD code. In the formula, σ(·) is the sigmoid function, T is the matrix transpose, and V=α T E, The attention mechanism is used to enable the interaction between E and C, specifically: Z = EC T The dimension of Z is ∑ L exp(Z) is the summation of exp(Z) along its Lth dimension.

[0042] j) By formula The cross-entropy loss function is calculated. In the formula, Y represents the manually proofread, accurate label corresponding to the input electronic medical record text. The value of label Y is either 0 or 1, and the dimension of label Y is... If the index corresponding to the value 1 in label Y is m, then m represents the current input electronic medical record text and the m-th ICD code that match, as shown by the formula. The overall loss was calculated. In the formula, λ is a hyperparameter. Preferably, in a specific embodiment of the present invention, λ = 0.0001, and the overall loss is reduced using the gradient descent method. Minimize the optimization of the multi-label classification model.

[0043] Due to the scarcity of high-quality electronic medical record data and corresponding ICD code annotation data, and the imbalance in sample size among different ICD code categories, general multi-label classification models suffer from poor learning performance for ICD code categories with limited sample sizes, resulting in poor classification performance for these categories. This method, through the gating mechanism in step g) and the constraint method in step h), dynamically controls the weight of the standard name of the ICD code in the vector representation of the ICD code when classifying categories with limited sample sizes. In step h), when k... m When it is very small, such as k m When = 5, optimize step j) and λ, making Minimizing λ will make k m =5 corresponds to the gating weight γ of the m-th ICD code. m Increase, and experiments have shown that when k increases m When = 5, ||γ m ||2≈9.5. When k m When it is very large, such as k m When = 1000, optimize step j) and λ, making Minimizing λ will make k m The gating weight γ corresponding to the m-th ICD code corresponding to 1000 m Decrease, as observed experimentally when k m When ||γ = 1000, ||γ m ||2≈2.5. Using this method, when the sample size for a certain ICD code category is insufficient, the model learns more vector representations from the standard name of that ICD code, thereby improving the underlearning problem caused by the scarcity of samples.

[0044] As attached Figure 2 and attached Figure 3 As shown in the attached figures, the horizontal axis of both figures represents the ICD coding category, arranged from most frequent to least frequent in the dataset. Figure 2 As shown, the smaller the value of τ, the higher the ||γ| of the ICD-coded categories with a sufficient sample size. m The larger ||2 is, the more ||γ m When ||2 is large, the model will learn more vector representations from the standard name of that ICD code. Experiments show that in the current application scenario, the model achieves optimal classification performance when τ = 1000.

[0045] To verify the effectiveness of the method, 20000 electronic medical records and their corresponding ICD codes manually annotated by human in a province of China are selected as the data set, the division of training set and test set is 70% and 30%, training on the training set, testing the classification performance of the model on the test set, comparing with the method of Mullenbach et al., the results of each evaluation standard in the following experimental results are the average values of the results of training 5 times of the two models using random initialization parameters:

[0046]

[0047] Table 2 Comparison of experimental results of existing methods (test set)

[0048] As shown in Table 2, the method of the present patent and the method of Mullenbach et al. are evaluated using AUC and F1 value and the accuracy of the top 8 ICD codes with the highest confidence predicted by the model, and it is found that the method of the present patent is better than the existing method in the three indicators.

[0049] To further verify the effectiveness of the method in the case of insufficient sample size, 20000 electronic medical records and their corresponding ICD codes manually annotated by human in a province of China are selected as the data set, and a subset containing 5000 electronic medical records is extracted from it, the division of training set and test set is 70% and 30%, training on the training set, testing the classification performance of the model on the test set, first using the method of the present patent to train the model, then using the method of Mullenbach et al. to train, and observing the training results using a small amount of samples, the results of each evaluation standard in the following experimental results are the average values of the results of training 5 times of the two models using random initialization parameters:

[0050]

[0051] Table 3 Experimental results on the subset containing 5000 electronic medical records (test set)

[0052] As shown in Table 3, by conducting experiments on the subset containing 5000 electronic medical records, it is found that the method of the present patent is better than the existing method on the small sample data set, and better classification performance can be achieved using fewer training samples than the existing method.

[0053] Further preferably, the matrix operation library in the implementation environment uses pytorch, and the hardware uses 4*NVIDIA GeForce RTX 3090 as GPU.

[0054] Example 1:

[0055] The values in the word vector matrix Δ in step a) are initialized from a random number obtained from represents a normal distribution with mean 0 and standard deviation 1.

[0056] Embodiment 2:

[0057] In step c), the vector representation of the standard name of the m-th ICD code after max-pooling is calculated by the formula In step c), the vector representation of the standard name of the m-th ICD code after max-pooling is calculated by the formula

[0058] Embodiment 3:

[0059] In step e), the values in the set U are initialized to random numbers obtained from In step e), the values in the set U are initialized to random numbers obtained from

[0060] Embodiment 4:

[0061] In step f), the gating weight Γ is calculated by the formula Γ = σ(QW2), where σ(·) is the sigmoid function, and W2 is a trainable parameter matrix, and the dimension of W2 is In step f), the gating weight Γ is calculated by the formula Γ = σ(QW2), where σ(·) is the sigmoid function, and W2 is a trainable parameter matrix, and the dimension of W2 is || is the matrix concatenation in the dimension of d, and the dimension of is |C| × 2d.

[0062] Embodiment 5:

[0063] In step g), the vector representation C of the ICD code is calculated by the formula In step g), the vector representation C of the ICD code is calculated by the formula The gating weight Γ weights the set and 1-Γ weights the set U, so that the model can use the gating mechanism to weight and fuse the sets and U when learning the vector representation C of the ICD code.

[0064] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An ICD disease classification oriented few-shot learning method, characterized in that, comprising the steps of: a) randomly initialize a word vector matrix Δ, for real space, V is the total number of words in the dictionary, and d is the dimension of the sub-vector. b) obtaining the electronic medical record text, a word vector matrix Δ maps each word in the electronic medical record text to a vector through the index in the dictionary, obtaining the word vector representation E of the electronic medical record text, E = {e1, e2,...,eL}, where e1 is the word vector corresponding to the i-th word in the electronic medical record text, e1 is a d-dimensional vector, i ∈ {1,...,L}, L is the total number of words in the electronic medical record text, i ,...,e L} where e i is the word vector corresponding to the i-th word in the electronic medical record text, e i is a d-dimensional vector, i ∈ {1,...,L}, L is the total number of words in the electronic medical record text, c) obtaining the vector representation of the standard name of each ICD code in the word vector representation E by querying the word vector matrix Δ, the vector representation of the standard name of the mthICD code is c m , is the word vector corresponding to the jthword in the standard name of the mthICD code, j ∈ {1,..., |m|}, |m| is the total number of words in the standard name of the mthICD code, m ∈ {1,..., |C|}, |C| is the number of ICD code categories in the word vector representation E, The vector representation c m of the standard name of the mthICD code is obtained by performing max-pooling in the dimension of |m| to obtain the vector representation of the standard name of the mthICD code after max-pooling d) constructing a set of all the vector representations of the ICD-coded standard names after maximum pooling obtained in step c) e) Initialize a trainable parameter vector for each ICD code, resulting in U = {u1, u2, ..., u...} m ,...,u |C| }, where u m Let m be the trainable parameter vector of the m-th ICD encoding, where m∈{1,...,|C|}. f) calculating a gating weight Γ, where γ m is the gating weight of the mthICD code, m ∈ {1,..., |C|}, γ m is a d-dimensional vector, The gating weight Γ takes a floating point number between 0 and 1, and does not include 0 and 1. g) calculating a vector representation C of the ICD code; h) establish a set K of all the occurrences of the codes in the data, K = {k1, k2,..., k m ,...,k |C|}, where kmis the number of occurrences of the mthICD code in the manually corrected ICD code dataset, m e {1,..., |C|}, and define a constrained loss function m m ,...,γ |C|} for Γ = {γ1, γ2,..., γ where || · ||2is the L2norm, max(K) is the maximum value in the set K, min(K) is the minimum value in the set K, and τ is a hyperparameter;​ i) by formula the probability of the current input electronic medical record text matching all |C| categories of ICD codes is calculated, the establishment of the multi-label classification model is completed, and the probability the value range is between 0 and 1 and does not include 0 and 1, and when the probability is greater than 0.5, the index corresponding to the value is m, then m is the multi-label classification model assigning the current input electronic medical record text to the mth ICD code, wherein σ(·) is a sigmoid function, T is a matrix transpose, V = α T E, Z = EC T , ∑ L exp(Z) is the Lth dimension of exp(Z) for summation calculation; j) By formula The cross-entropy loss function is calculated. In the formula, Y represents the manually verified accurate label corresponding to the input electronic medical record text. The value of label Y is either 0 or 1. If the index corresponding to the value 1 in label Y is m, then m represents the current input electronic medical record text and the m-th ICD code that match, as shown by the formula. The overall loss was calculated. In the formula, λ is a hyperparameter, and the overall loss is calculated using gradient descent. Minimize the optimization of the multi-label classification model.

2. The ICD disease classification oriented few-shot learning method according to claim 1, wherein: The values in the vector matrix Δ in step a) are initialized to random numbers taken from denotes a normal distribution with mean 0 and standard deviation 1.​ 3. The ICD disease classification oriented few-shot learning method according to claim 1, wherein: The vector representation of the standard name of the mthICD code after max-pooling is calculated in step c) by the formula The vector representation of the standard name of the mthICD code after max-pooling is calculated in step c) by the formula 4. The ICD disease classification oriented few-shot learning method according to claim 2, characterized in that: The values in the set U in step e) are initialized to random numbers taken from the set of random numbers.

5. The ICD disease classification oriented few-shot learning method according to claim 1, wherein: The gating weight G is calculated in step f) by the formula G = σ(QW2), where σ(·) is a sigmoid function and W2is a matrix of trainable parameters, where W1is a matrix of trainable parameters, || is a matrix concatenation in the dimension of d, has a dimension of |C| x 2d.

6. The ICD disease classification oriented few-shot learning method according to claim 1, wherein: The vector representation C of the ICD code is computed in step g) by the formula ​ 7. The ICD disease classification oriented few-shot learning method according to claim 1, characterized in that: In step h) τ = 1000.

8. The ICD disease classification oriented few-shot learning method according to claim 1, characterized in that: In step j) λ = 0.0001. In step j) λ = 0.0001.

Images