Automatic ICD9 code assignment method for medical record text based on label attributes and feature enhancement

Abstract

The invention discloses an automatic allocation method of medical record text ICD9 codes based on label attributes and feature enhancement. The ICD9 code is regarded as a label, and its corresponding text description is used as the label attribute. The label attribute feature and the semantic feature of each word in the medical record text are obtained by semantic encoding of the label attribute and the medical record text; a relevant content extraction vector and a label semantic vector are set for each label, and the relevant content and semantic features are extracted from the medical record text by using the extraction vectors respectively, and the extracted relevant features are enhanced by using the co-occurrence relationship; using the attention mechanism, for different labels, the extracted multiple relevant content semantic features, label attributes and semantic features are adaptively fused to obtain the final relevant content semantics and label semantic representation; the relevant content semantic features are compared with the label semantics to obtain the classification result, and according to the hierarchical relationship between the ICD9 codes, layer-by-layer negative sampling, parameter training and prediction are performed.

Description

Technical Field

[0001] The invention relates to a method for automatically allocating ICD9 codes of medical record text based on label attributes and feature enhancement. Background Art

[0002] The content of the medical record is the diagnosis and treatment record of the patient during the medical treatment process. Among them, the ICD9 code corresponds to the disease and surgery. A patient may have multiple diseases and undergo multiple surgeries during a diagnosis and treatment process. The automatic allocation of ICD9 codes for medical record text is to automatically allocate the corresponding ICD9 codes to the medical record text. Among them, one medical record text can be assigned multiple ICD9 codes.

[0003] In the existing methods, most of them regard each ICD9 code as a label, and regard the automatic assignment of ICD9 codes to medical record text as a text multi-label classification problem, and judge whether each ICD9 code is related to the medical record text one by one. For example, in the first method, 1 relevant content extraction vector and 1 semantic vector are set for each ICD9 code. For each ICD9 code, the content extraction vector is used to extract relevant content from the medical record text to obtain its semantic representation, and then compared with the semantic vector of the corresponding ICD9 code to obtain the classification result. In addition, the ICD9 code description text is semantically encoded, and the encoding result is as close as possible to the semantic vector during training. During testing, the encoding result is used as the semantic vector for ICD9 codes that do not appear in the training set. In the second method, the description text of the ICD9 code is first encoded to obtain the content extraction vector of the ICD9 code. On this basis, the co-occurrence and hierarchical relationship between the ICD9 codes are further combined to obtain the semantic vector of the ICD9 code, and then the relevant content is extracted and classified for each ICD9 code. In the third method, the encoding results of the ICD9 description text and co-occurrence relationship are first used as the relevant content extraction vector, and an additional relevant content extraction vector is defined for each ICD9 code; then, for each ICD9 code, two vectors are used to simultaneously extract relevant content from the medical record text to obtain its semantic feature representation and splice it; finally, the splicing result is compared with the corresponding ICD9 code semantic vector to obtain the classification result. When classifying, it also utilizes the hierarchical structure of the ICD9 code, and classifies each layer from top to bottom. The classification result of the previous layer is the basis for the classification of the next layer.

[0004] However, first of all, the descriptive information of the medical record text, that is, the label attribute, plays an important role in extracting relevant content from the medical record text and the semantic representation of the ICD9 code. However, the limited content in the description of the ICD9 code may not be enough to fully and accurately represent the meaning of the ICD9 code. It is necessary to additionally define the relevant content extraction vector and the ICD9 code semantic vector, and cooperate with them respectively to more accurately extract relevant content and represent the ICD9 code semantics. Among the above methods, only the third one takes this situation into consideration when extracting relevant content from the medical record text, and all methods do not take this into consideration when representing the semantics of the ICD9 code.

[0005] Secondly, different ICD9 codes, that is, different diseases and surgeries, are related to each other. To determine whether a medical record text corresponds to an ICD9 code, the corresponding content of the relevant ICD9 code in the medical record text is crucial. Existing methods only use the relationship between ICD9 codes and the label attributes of ICD9 codes to enhance each other's features, but do not enhance the features of the relevant content features extracted from different ICD9 codes.

[0006] Secondly, the ICD9 code shows an obvious long-tail distribution phenomenon in the training data set. Some ICD9 codes have very little training data corresponding to them, and the parameters are difficult to learn. Existing methods do not consider how to reduce the dependence on training data during parameter learning.

[0007] Finally, for different ICD9 codes, the effects of label attributes and feature enhancement vary. For example, for some ICD9 codes with very little training data, it is difficult to obtain the accurate semantics of the ICD9 code through training data, so more label attribute information is needed; the description text of some ICD9 codes is very different from the actual data, so the role of label attributes is very small; some ICD9 codes require more relationships with other ICD9 codes to obtain accurate classification results. It is necessary to adaptively integrate these aspects of information according to the specific meaning of the ICD9 code, which is not considered by existing methods. Although Method 3 represents the relevant content semantic features obtained by label attributes and content extraction vectors for all ICD9 codes, it only uses a simple splicing method. Summary of the invention

[0008] The purpose of the present invention is to provide a method for automatic assignment of ICD9 codes to medical record texts based on label attributes and feature enhancement, which can more accurately and comprehensively obtain relevant content and semantic features of ICD9 codes, better cope with the long-tail distribution of ICD9 codes, and greatly improve the accuracy and recall rate of automatic assignment of ICD9 codes.

[0009] In order to achieve the above object, the present invention provides a method for automatically assigning ICD9 codes to medical record text based on label attributes and feature enhancement, comprising:

[0010] Step 1, obtain data from the Internet, including medical record text and its corresponding ICD9 code, ICD9 code description text, and ICD9 code hierarchy; remove words that appear less than 2 times and more than 3000 times in the medical record text and ICD9 code description text, and process the length of the medical record text into N, truncate those greater than a fixed length, and fill in "PAD" strings less than a fixed length; randomly select M medical records from the acquired data as a training data set, and store all the words in the dictionary, where the number of words is V;

[0011] Step 2: Use the Gensim tool to train Skip-gram word vectors based on all the training set medical records and ICD9 code description texts to obtain the d-dimensional vector representation corresponding to each word, forming a word vector matrix E∈R V×d , take E as the word vector matrix E corresponding to the ICD9 description text C ∈R V×d , the word vector matrix E corresponding to the medical record text D ∈R V×d The initial value of

[0012] Step 3: According to the hierarchical structure of ICD9 codes, ICD9 codes are divided into 4 groups: 1 , C 2 , C 3 , C 4 ; C 4 The set of ICD9 codes corresponding to the medical record text in all data is obtained Represents C 4 ICD9 code and C 3 The relationship between ICD9 codes; Represents C 4 The i-th ICD9 code belongs to C 3 The jth ICD9 code in the same way is used to obtain I 3 ,I 2 ;

[0013] Step 4: Get C through the training data set 4 Correspondence Matrix Represents C 4 The j-th ICD9 code in is assigned to the i-th case in the training set; get

[0014] Step 5: Generate C using the ICD9 code co-occurrence matrix generation algorithm 4 ICD9 code co-occurrence matrix corresponding to the set In the algorithm, ω and f are positive integers, p∈[0,1]; on this basis, Get G 3 , G 2 , G 1 ;

[0015] Step 6: Represent each medical record text as a matrix consisting of word one-hot vectors When the nth position of the ith document is the vth word otherwise C 4 The descriptive text of each ICD9 code is represented as a V-dimensional vector S c,v The value of is the number of times the vth word appears in the description text of the cth ICD9 code; the description text corresponding to each ICD9 code in the other sets is represented as Where ch(c) represents the direct child node of node c;

[0016] Step 7: M medical records and their corresponding Construct a training data set, randomly divide the training data set into several batches of size B, and perform parameter training in batches;

[0017] Step 8: Repeat step 7 for 10-20 rounds to obtain the values ​​of each parameter and the classification results of each medical record text in the last round. And save it persistently;

[0018] Step 9: Collect the medical records and their corresponding Construct a training data set for C 2 Parameter learning is performed on the ICD9 code in the medical record text; due to the long-tail distribution of medical record text, some ICD9 codes have very little corresponding training data and parameters are difficult to train. 1 The classification results in C 2 The code in the code performs negative sampling; first, Sort from high to low, take the first Z 1 The set C(i) is formed, and the set Among them, ch(C(i)) represents the direct child nodes of all nodes of C(i); then, use G 2 , C 2 , Replace the original G 1 , C 1 , Re-execute steps 7 and 8; when re-executing step 7, set the initial values ​​of each parameter to the values ​​obtained in step 8; after re-executing step 8, new parameters will be obtained. And save it persistently;

[0019] Step 10: Collect the medical records and their corresponding Construct a training data set for C 3 ICD9 code in the parameter learning; first, Sort from high to low, take the first Z 2 The set C(i) is formed, and the set Then, use G 3 , C 3 , Replace the original G 1 , C 1 , Re-execute step 7 and step 8; when re-executing step 7, set the initial value of each parameter to the parameter value obtained in step 9; re-execute step 8 to obtain new parameters, And save it persistently;

[0020] Step 11: Collect the medical records and their corresponding Construct a training data set for C 4 ICD9 code in the parameter learning; first, Sort from high to low, take the first Z 3 The set C(i) is formed, and the set Then, use G 4 , C 4 , Replace the original G 1 , C 1 , Re-execute step 7 and step 8; when re-executing step 7, set the initial value of each parameter to the parameter value obtained in step 10; re-execute step 8 to obtain new parameters, And save it persistently;

[0021] Step 12: Use the parameters obtained in steps 8, 9, 10, and 11 and C 1 , G 1 , C 2 , G 2 , C 3 , G 3 , C 4 , G 4 Initialize the calculation process in step 7, Replace with C1 , C 2 , C 3 , C 4 , get the value for C 1 , C 2 , C 3 , C 4 Classification model of

[0022] Step 13: For each test data, execute step 6 first, and then send it to the C 1 , C 2 , C 3 , C 4 The classification model assigns the U corresponding to the ICD9 code that has never been seen c 、V c Set to 0 d , and obtain the classification results of 4 models Will The ICD9 codes are assigned to the corresponding medical records.

[0023] Preferably, step 7 comprises:

[0024] Step 7.1: Use the description text of the ICD9 code as the label attribute of the ICD9 code. 1 For each ICD9 code, Get the label attribute characteristics of each ICD9 code

[0025] Step 7.2: For C 1 For each ICD9 code in Perform feature extraction to obtain a new representation of label attribute features in, is the parameter matrix;

[0026]

[0027] Step 7.3: Target C 1 Each ICD9 code in C 1 The co-occurrence relationship between ICD9 codes, that is, the co-occurrence matrix G 1 , use (2) and (3) to calculate the attribute features Perform feature enhancement to obtain enhanced label attribute features in, is the parameter matrix;

[0028]

[0029]

[0030] Step 7.4: Target C 1 Each ICD9 code in U c Perform adaptive fusion, U c ∈R d is the semantic vector of the cth ICD9 code defined additionally, and its initial value is

[0031] The fusion calculation process is as follows: (4), (5), (6) and (7), where T W ∈R d×d , A W ∈R 3×3d is the parameter matrix, Represents vector concatenation, q∈R 3d ,q′∈R 3 Indicates the proportion of the three parts; each ICD9 code will determine the fusion ratio according to its own characteristics, and obtain the ICD9 code semantic representation W c ;

[0032] t(x)=sigmoid(T W ReLU(x)) (4)

[0033]

[0034]

[0035]

[0036] Step 7.5: Perform parameter training for each medical record in the batch;

[0037] Step 7.6: Calculate the cross entropy loss corresponding to the batch according to formula (24), where Indicates whether the i-th medical record text in the training set is assigned the c-th ICD9 code;

[0038]

[0039] Step 7.7: Use the Adam optimization algorithm to adjust the parameter E according to the loss. C , U c , T W , A W 、E D , {CNN l,k} l＝1:L,k＝1:K , T , R T , V c 、T H , AH Learning is performed with a learning rate of μ.

[0040] Preferably, step 7.5 includes:

[0041] Step 7.5.1. Pass Get the word vector representation of the medical record text content Will The medical record text is sent to a multi-layer convolutional network for feature extraction. The calculation process is as follows: (8) and (9); lk represents the kth CNN in the lth layer, CNN :,k The corresponding convolution kernel size is size(k); all CNN l,k The movement amplitude is 1, the number of input channels is d, and the number of output channels is 2*d; pad k () The calculation process is:

[0042] If size(k) is not divisible by 2, or Add (size(k)-1) / 2 0 vectors before and after the first dimension of d ;

[0043] If size(k) is divisible by 2, then or Add (size(k)-1) / 2 zeros to the first dimension of d , add size(k) / 2 0s to the end d ;

[0044] After each CNN l,k Then we will get the representation of each text L×K CNNs lk All semantic representations of each medical record text will be obtained

[0045]

[0046]

[0047] Step 7.5.2: For the i-th medical record text For each ICD9 code in from Get the closely related content and its semantic representation Will All are set to 0 d , the calculation process is as follows: (10) and (11); where λ T ∈[0,1], R T ∈Rd×d is the parameter matrix, The calculation process of is as follows: (12), (13), (14) and (15); where: represents all semantic vectors of the nth word in the i-th medical record text obtained in step 7.5.1, a i,c,n,: ∈R (L×K) Indicates the closeness between the c-th ICD9 code and the different semantic representations of the n-th word in the ith medical record text; represents the semantic representation of the nth word obtained by comparing the i-th medical record text with the c-th ICD9 code, β i，c,: ∈R N Indicates the closeness between the cth ICD9 code and different words in the i-th medical record text;

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] Step 7.5.3: For each ICD9 code in Perform feature extraction and obtain

[0055]

[0056] Step 7.5.4: For each ICD9 code in, according to the co-occurrence relationship between ICD9 codes, that is, the co-occurrence matrix G 1 , using equations (17) and (18) to Perform feature enhancement to obtain

[0057]

[0058]

[0059] Step 7.5.5: For each ICD9 code, additional Extract relevant content, V c ∈R dExtract the vector corresponding to the relevant content of the cth ICD9 code, with the initial value being From another perspective, we can obtain the content and semantic representation related to the cth ICD9 code in the i-th medical record text. The calculation process of is as follows: (12), (13), (14) and (15);

[0060] Step 7.5.6: For each ICD9 code in Adaptively fuse and obtain the semantic representation H of the final i-th medical record text and the c-th ICD9 code related content i,c ∈R d ; The calculation process is as follows: (19), (20), (21) and (22), where T H ∈R d×d , A H ∈R 3×3d is a parameter matrix, q′ represents the fusion ratio of the three parts, and will have different values ​​for different ICD9 codes;

[0061] t(x)=sigmoid(T H ReLU(x)) (19)

[0062]

[0063]

[0064]

[0065] Step 7.5.7: For each ICD9 code in H, use formula (23) to i，c With W c By comparison, the classification result y is obtained i，c ∈[0, 1], y i,c represents the probability that the i-th medical record text is assigned the c-th ICD9 code; par(c) represents the parent node of the c-th ICD9 code, y i,par(c) Indicates that the i-th medical record text is the comparison result of par(c); if par(c)=φ, y i，par(c) =1;

[0066] y i，c =y i，par(c) sigmoid(W c H i，c T ) (twenty three).

[0067] According to the above technical scheme, firstly, the ICD9 code is regarded as a label, and the text description corresponding to the ICD9 code is regarded as the label attribute, and the label attribute features and the semantic features of each word in the medical record text are obtained by semantic encoding of the label attributes and the medical record text; then, a relevant content extraction vector and a label semantic vector are set for each label, and the label attribute features and the relevant content extraction vector are used to extract relevant content and its semantic features from the medical record text respectively, and the co-occurrence relationship between different ICD9 codes is used to enhance the features of the extracted relevant content semantic features and label attribute features; secondly, the attention mechanism is used to adaptively fuse the extracted multiple relevant content semantic features, label attributes and semantic features for different labels to obtain the final relevant content semantic and label semantic representation; finally, the relevant content semantic features are compared with the label semantics to obtain the classification results, and according to the hierarchical relationship between the ICD9 codes, layer-by-layer negative sampling, parameter training and prediction are performed.

[0068] Other features and advantages of the present invention will be described in detail in the following detailed description. DETAILED DESCRIPTION

[0069] The specific embodiments of the present invention are described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0070] The present invention provides a method for automatically allocating ICD9 codes of medical record text based on label attributes and feature enhancement, comprising:

[0071] Step 1, obtain data from the Internet, including medical record text and its corresponding ICD9 code, ICD9 code description text, and ICD9 code hierarchy; remove words that appear less than 2 times and more than 3000 times in the medical record text and ICD9 code description text, and process the length of the medical record text into N, truncate those greater than a fixed length, and fill in "PAD" strings less than a fixed length; randomly select M medical records from the acquired data as a training data set, and store all the words in the dictionary, where the number of words is V;

[0072] Step 2: Use the Gensim tool to train Skip-gram word vectors based on all the training set medical records and ICD9 code description texts to obtain the d-dimensional vector representation corresponding to each word, forming a word vector matrix E∈R V×d , take E as the word vector matrix E corresponding to the ICD9 description text C ∈R V×d , the word vector matrix E corresponding to the medical record text D ∈R V×d The initial value of

[0073] Step 3: According to the hierarchical structure of ICD9 codes, ICD9 codes are divided into 4 groups: 1 , C 2 , C 3 , C 4 ; C 4 The set of ICD9 codes corresponding to the medical record text in all data is obtained Represents C 4 ICD9 code and C 3 The relationship between ICD9 codes; Represents C 4 The i-th ICD9 code belongs to C 3 The jth ICD9 code in the same way is used to obtain I 3 ,I 2 ;

[0074] Step 4: Get C through the training data set 4 Correspondence Matrix Represents C 4 The j-th ICD9 code in is assigned to the i-th case in the training set; get

[0075] Step 5: Generate C using the ICD9 code co-occurrence matrix generation algorithm 4 ICD9 code co-occurrence matrix corresponding to the set In the algorithm, ω and f are positive integers, p∈[0,1]; on this basis, Get G 3 , G 2 , G 1 ;

[0076] Among them, the ICD9 code co-occurrence matrix generation algorithm is as follows:

[0077]

[0078] Step 6: Represent each medical record text as a matrix consisting of word one-hot vectors When the nth position of the ith document is the vth word otherwise C 4 The descriptive text of each ICD9 code is represented as a V-dimensional vector S c，v The value of is the number of times the vth word appears in the description text of the cth ICD9 code; the description text corresponding to each ICD9 code in the other sets is represented as Where ch(c) represents the direct child node of node c;

[0079] Step 7: M medical records and their corresponding Construct a training data set, randomly divide the training data set into several batches of size B, and perform the following process in batches:

[0080] Step 7.1: As the description text plays an important role in describing the semantics of ICD9 codes, the description text of ICD9 codes is used as the label attribute of ICD9 codes. 1 For each ICD9 code, Get the label attribute characteristics of each ICD9 code

[0081] Step 7.2: For C 1 For each ICD9 code in Perform feature extraction to obtain a new representation of label attribute features in, is the parameter matrix;

[0082]

[0083] Step 7.3: Since different ICD9 codes are related to each other, the attributes of related ICD9 codes can effectively enrich the semantic representation of the corresponding ICD9 codes. 1 Each ICD9 code in C 1 The co-occurrence relationship between ICD9 codes, that is, the co-occurrence matrix G 1 , use (2) and (3) to calculate the attribute features Perform feature enhancement to obtain enhanced label attribute features in, is the parameter matrix;

[0084]

[0085]

[0086] Step 7.4: Since label attributes cannot fully and accurately reflect the semantics of ICD9 codes, and the effects of label attributes on different ICD9 codes are also different, 1 Each ICD9 code in U c Perform adaptive fusion, U c ∈R d is the semantic vector of the cth ICD9 code defined additionally, and its initial value is

[0087] The fusion calculation process is as follows: (4), (5), (6) and (7), where T W ∈R d×d , A W ∈R 3×3d is the parameter matrix, Represents vector concatenation, q∈R 3d ,q′∈R 3 Indicates the proportion of the three parts; each ICD9 code will determine the fusion ratio according to its own characteristics, and obtain the ICD9 code semantic representation W c ;

[0088] t(x)=sigmoid(T W ReLU(x)) (4)

[0089]

[0090]

[0091]

[0092] Step 7.5: Perform parameter training for each medical record in the batch; the specific execution process is as follows:

[0093] Step 7.5.1. Pass Get the word vector representation of the medical record text content Will The medical record text is sent to a multi-layer convolutional network for feature extraction. The calculation process is as follows: (8) and (9); lk represents the kth CNN in the lth layer, CNN :,k The corresponding convolution kernel size is size(k); all CNN l,k The movement amplitude is 1, the number of input channels is d, and the number of output channels is 2*d; pad k () The calculation process is:

[0094] If size(k) is not divisible by 2, or Add (size(k)-1) / 2 0 vectors before and after the first dimension of d ;

[0095] If size(k) is divisible by 2, then or Add (size(k)-1) / 2 zeros to the first dimension of d , add size(k) / 2 0s to the end d ;

[0096] After each CNNl,k Then we will get the representation of each text L×K CNNs lk All semantic representations of each medical record text will be obtained

[0097]

[0098]

[0099] Step 7.5.2: For the i-th medical record text For each ICD9 code in from Get the closely related content and its semantic representation Will All are set to 0 d , the calculation process is as follows: (10) and (11); where λ T ∈[0,1], R T ∈R d×d is the parameter matrix, The calculation process of is as follows: (12), (13), (14) and (15); where represents all semantic vectors of the nth word in the i-th medical record text obtained in step 7.5.1, a i,c,n,: ∈R (L×K) Indicates the closeness between the c-th ICD9 code and the different semantic representations of the n-th word in the ith medical record text; represents the semantic representation of the nth word obtained by comparing the i-th medical record text with the c-th ICD9 code, β i,c,: ∈R N Indicates the closeness between the cth ICD9 code and different words in the i-th medical record text;

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106] Step 7.5.3: For each ICD9 code in Perform feature extraction and obtain

[0107]

[0108] Step 7.5.4: Since the relevant ICD9 codes correspond to the content in the medical record text, it is very important to determine whether an ICD9 code should be assigned to a medical record text. For each ICD9 code in, according to the co-occurrence relationship between ICD9 codes, that is, the co-occurrence matrix G 1 , use (17) and (18) to Perform feature enhancement to obtain

[0109]

[0110]

[0111] Step 7.5.5: Since the text description of ICD9 codes may not fully and accurately describe the semantics of ICD9 codes, For each ICD9 code, additional Extract relevant content, V c ∈R d Extract the vector corresponding to the relevant content of the cth ICD9 code, with the initial value being From another perspective, we can obtain the content and semantic representation related to the cth ICD9 code in the i-th medical record text. The calculation process of is as follows: (12), (13), (14) and (15);

[0112] Step 7.5.6: Due to the different ICD9 codes, the tag attributes have different effects. For each ICD9 code in Adaptively fuse and obtain the semantic representation H of the final i-th medical record text and the c-th ICD9 code related content i,c ∈R d ; The calculation process is as follows: (19), (20), (21) and (22), where T H ∈R d×d , A H ∈R 3×3d is a parameter matrix, q′ represents the fusion ratio of the three parts, and will have different values ​​for different ICD9 codes;

[0113] t(x)=sigmoid(T H ReLU(x)) (19)

[0114]

[0115]

[0116]

[0117] Step 7.5.7: For each ICD9 code in H, use formula (23) to i,c With W c By comparison, the classification result y is obtained i,c ∈[0,1],y i,c represents the probability that the i-th medical record text is assigned the c-th ICD9 code; par(c) represents the parent node of the c-th ICD9 code, y i,par(c) Indicates that the i-th medical record text is the comparison result of par(c); if par(c)=φ, y i,par(c) =1;

[0118] y i,c =y i,par(c) sigmoid(W c H i,c T ) (twenty three).

[0119] Step 7.6: Calculate the cross entropy loss corresponding to the batch according to formula (24), where Indicates whether the i-th medical record text in the training set is assigned the c-th ICD9 code;

[0120]

[0121] Step 7.7: Use the Adam optimization algorithm to adjust the parameter E according to the loss. C , U c , T W , A W 、E D , {CNN l,k} l＝1:Lk＝1:K , T , R T , V c , T H , A H Learning is performed with a learning rate of μ.

[0122] Step 8: Repeat step 7 for 10-20 rounds to obtain the values ​​of each parameter and the classification results of each medical record text in the last round. And save it persistently;

[0123] Step 9: Collect the medical records and their corresponding Construct a training data set for C 2 Parameter learning is performed on the ICD9 code in the medical record text; due to the long-tail distribution of medical record text, some ICD9 codes have very little corresponding training data and parameters are difficult to train. 1 The classification results in C 2 The code in the code performs negative sampling; first, Sort from high to low, take the first Z 1 The set C(i) is formed, and the set Among them, ch(C(i)) represents the direct child nodes of all nodes of C(i); then, use G 2 , C 2 , Replace the original G 1 , C 1 , Re-execute steps 7 and 8; when re-executing step 7, set the initial values ​​of each parameter to the values ​​obtained in step 8; after re-executing step 8, new parameters will be obtained. And save it persistently;

[0124] Step 10: Collect the medical records and their corresponding Construct a training data set for C 3 ICD9 code in the parameter learning; first, Sort from high to low, take the first Z 2 The set C(i) is formed, and the set Then, use G 3 , C 3 , Replace the original G 1 , C 1 , Re-execute step 7 and step 8; when re-executing step 7, set the initial value of each parameter to the parameter value obtained in step 9; re-execute step 8 to obtain new parameters, And save it persistently;

[0125] Step 11: Collect the medical records and their corresponding Construct a training data set for C 4 ICD9 code in the parameter learning; first, Sort from high to low, take the first Z 3 The set C(i) is formed, and the set Then, use G 4 , C 4 , Replace the original G 1, C 1 , Re-execute step 7 and step 8; when re-executing step 7, set the initial value of each parameter to the parameter value obtained in step 10; re-execute step 8 to obtain new parameters, And save it persistently;

[0126] Step 12: Use the parameters obtained in steps 8, 9, 10, and 11 and C 1 , G 1 , C 2 , G 2 , C 3 , G 3 , C 4 , G 4 Initialize the calculation process in step 7, Replace with C 1 , C 2 , C 3 , C 4 , get the value for C 1 , C 2 , C 3 , C 4 Classification model of

[0127] Step 13: For each test data, execute step 6 first, and then send it to the C 1 , C 2 , C 3 , C 4 The classification model assigns the U corresponding to the ICD9 code that has never been seen c 、V c Set to 0 d , and obtain the classification results of 4 models Will The ICD9 codes are assigned to the corresponding medical records text.

[0128] It can be seen that in order to obtain the semantic representation of relevant content and ICD9 code more accurately and comprehensively, the present invention uses the description text of ICD9 code, i.e., label attribute, in the process of obtaining the semantics of relevant content and ICD9 code, and utilizes the co-occurrence relationship between ICD9 codes to enhance the extracted semantic features of relevant content and category attribute features respectively;

[0129] Secondly, in order to better deal with the long-tail distribution phenomenon of ICD9 codes, for different ICD9 codes, the attention mechanism is used to adaptively fuse the content semantic features and label semantic features obtained from multiple different angles, and the negative sampling method is used for label training layer by layer to improve the training effect of ICD9 codes with less training data.

[0130] In a specific embodiment of the present invention,

[0131] Step 1: Get data from https: / / mimic.mit.edu / . The data report has 5 CSV tables: NOTEEVENTS.csv includes the medical records of all patients, DIAGNOSES_ICD.csv contains all the diagnosis-related ICD9 codes corresponding to each medical record text, PROCEDURES_ICD.csv contains all the surgery-related ICD9 codes corresponding to each medical record text, D_ICD_DIAGNOSES.csv and D_ICD_PROCEDURES.csv contain the text descriptions of all ICD9 codes, and the hierarchical structure of ICD9 codes is obtained from https: / / www.cdc.gov / nchs / icd / icd9cm.htm. It is stored in 3 columns. The first column is the ICD9 code of the child node, the second column is the code of the corresponding parent node, and the third column is the level (1, 2, 3, 4) corresponding to the ICD9 code of the child node. The parent node of the first-level ICD code is "φ". Words with a frequency greater than 1 and less than 3,000 in the medical record text and ICD9 code description text were removed, added to the dictionary and numbered, and a total of 150,229 words were included. The length of the medical record text was processed to 4,000, and words greater than the fixed length were truncated, and words less than the fixed length were padded with a "PAD" string. Words not in the dictionary were processed as "myunk". At this time, the pandas[4] toolkit was used to process the acquired data into the following format, where TEXT is the medical record text and LABELS is the corresponding ICD9\code. 47,724 medical records were randomly selected from the acquired data as the training data set.

[0132]

[0133] Step 2: Use the Gensim tool to train Skip-gram word vectors based on all the training set medical records and ICD9 code description texts. Set d to 100 and the number of training rounds to 10 to obtain the 100-dimensional vector representation corresponding to each word, forming the word vector matrix E∈R 150229×100 .

[0134] Step 3: Divide the ICD9 codes into 4 groups according to the hierarchy of ICD9 codes 1 , C 2 , C 3 , C 4 , where C 4 For the ICD9 code set of the 4th layer, get the affiliation relationship between ICD9 codes Among them, |C 4|、|C 3 |、|C 2 |、|C 1 |The sizes are 8922, 4755, 1100, and 155 respectively.

[0135] Step 4: Get C through the training data set 4 Correspondence Matrix Represents C 4 The j-th ICD9 code in is assigned to the i-th case in the training set. get

[0136] Step 5: Generate C using the ICD9 code co-occurrence matrix generation algorithm 4 ICD9 code co-occurrence matrix corresponding to the set Set ω, p, and f in the algorithm to 400, 0.8, and 800, respectively. Get G 3 , G 2 , G 1 .

[0137] Step 6: Represent each medical record text as a matrix consisting of word one-hot vectors C 4 The descriptive text of each ICD9 code is represented as a V-dimensional vector Represent each ICD9 code in the other set as ∑S t∈ch(c) . ch(c) represents the direct child node of node c.

[0138] Step 7: Use pytorch to convert the medical record text and the corresponding Construct a training data set, randomly divide the training data set into several batches of size 32, and perform the following process in batches:

[0139] Step 7.1: C 1 Each ICD9 code is fed into the word vector layer Get its label attribute characteristics The word embedding layer is implemented using pytorch’s torch.nn.Embedding. The num_embeddings and embedding_dim of Embedding are set to 150229 and 100 respectively, and the initial value of the parameter is set to E.

[0140] Step 7.2: Targeting C 1 For each ICD9 code in Perform feature extraction to obtain a new representation W of the label attribute featurec 1 ∈R 100 ,in is the parameter matrix. The ReLU function is implemented using the nn.functional.relu function in pytorch.

[0141] Step 7.3: Targeting C 1 Each ICD9 code in C 1 The co-occurrence relationship between ICD9 codes, that is, the co-occurrence matrix G 1 , use (2) (3) to identify the label attribute features Perform feature enhancement to obtain enhanced label attribute features

[0142] Section 7.4: Specific to C 1 Each ICD9 code in U c The calculation process is (4)(5)(6)(7). The sigmoid function is implemented by nn.functional.sigmoid function of pytorch. This is implemented by pytorch's torch.cat function. Get C 1 The semantic representation of each ICD9 code in W c ∈R 100 .

[0143] Step 7.5: Perform the following process for each medical record text in the batch:

[0144] Step 7.5.1: Send it to the word vector layer corresponding to the medical record text. Get the word vector representation of the medical record text content The word embedding layer is implemented by torch.nn.Embedding of pytorch. The num_embeddings and embedding_dim of Embedding are set to 150229 and 100 respectively, and the initial value of the parameter is set to E. The medical record text is sent to a multi-layer convolutional network for feature extraction. The calculation process is as follows: (1) (2). Where l and k are 1, 2, 3, 4. All CNN l,k All are implemented through pytorch's torch.nn.Conv1d layer. Each CNN l,k Corresponding to 1 Conv1d, the in_channels, out_channels, and stride of all Conv1d are set to 100, 200, and 1 respectively. CNN l,1,CNN l,2 ,CNN l,3 ,CNN l,4 The kernel_size values ​​are 3, 4, 5, and 6 respectively. The pad function is implemented by pytorch's nn.functional.pad function, and the GLU is implemented by pytorch's nn.functional.glu function. After each CNN, we get 4×4 CNNs will obtain the matrix representation of each medical record text

[0145] Step 7.5.2: For the i-th medical record text For each ICD9 code in from Find the content closely related to it in Semantic representation of all ICD9 codes in the medical record text Will All are set to 0 100 The calculation process is as follows (10)(11), where the softmax function is implemented by torch.nn.Softmax of pytorch.

[0146] Step 7.5.3: Target For each ICD9 code in Perform feature extraction and obtain

[0147] Step 7.5.4: Target For each ICD9 code in, according to the co-occurrence matrix G 1 , use (17) (18) to calculate each content feature Perform feature enhancement to obtain

[0148] Step 7.5.5: Target For each ICD9 code, The calculation process is as follows: (12)(13)(14)(15), from another perspective, we can obtain the content features related to the cth ICD9 code in the i-th medical record text:

[0149] Step 7.5.6: Target For each ICD9 code, (19)(20)(21)(22) Adaptively fusion is performed to obtain the semantic representation H of the content related to the cth ICD9 code in the i-th medical record text i,c ∈R 100.

[0150] Step 7.5.7: Target For each ICD9 code in H, use formula (23) to i,c With W c By comparison, the classification result y is obtained i,c ∈[0,1].y i,c represents the probability that the i-th medical record text is assigned the c-th ICD9 code. par(c) represents the parent node of the c-th ICD9 code, y i,par(c) Indicates the comparison result of par(c) for the i-th medical record text. If par(c)=φ, y i,par(c) =1.

[0151] Step 7.6: Implement equation (24) through the nn.functional.binary_cross_entropy_with_logits function to calculate the cross entropy loss corresponding to the batch.

[0152] Step 7.7: Use pytorch's torch.optim.Adam to implement the Adam optimization algorithm, set the Adam learning rate lr to 0.2, and adjust the parameter E according to the loss C , U c 、T W , A W , E D , {CNN l,k} l＝1:L,k＝1:K , T , R T , V c 、T H , A H study.

[0153] Step 8: Repeat step 7 for 20 rounds to obtain the values ​​of each parameter and the classification results of each medical record text in the last round. And save it persistently.

[0154] Step 9: Combine the case texts in the training set and the corresponding Construct a training data set for C 2 First, we need to learn the parameters of ICD9 code. Sort from high to low, take the first Z 1 = 100 to form a set C(i), and further obtain the set Then use G 2 , C 2 , Replace the original G1 , C 1 , Re-execute steps 7 and 8. When re-executing step 7, set the initial values ​​of each parameter to the values ​​obtained in step 8. After re-executing step 8, new parameters will be obtained. And save it persistently.

[0155] Step 10: Collect the medical records and their corresponding Construct a training data set for C 3 First, we need to learn the parameters of ICD9 code. Sort from high to low, take the first Z 2 = 600 to form the set C(i), and further obtain the set Then use G 3 , C 3 , Replace the original G 1 , C 1 , Re-execute steps 7 and 8. When re-executing step 7, set the initial values ​​of each parameter to the parameter values ​​obtained in step 9. Re-execute step 8 to obtain new parameters. And save it persistently.

[0156] Step 11: Collect the medical records and their corresponding Construct a training data set for C 4 First, we need to learn the parameters of ICD9 code. Sort from high to low, take the first Z 3 = 1000 to form the set C(i), and further obtain the set Then use G 4 , G 4 , Replace the original G 1 , C 1 , Re-execute steps 7 and 8. When re-executing step 7, set the initial values ​​of each parameter to the parameter values ​​obtained in step 10. Re-execute step 8 to obtain new parameters. And save it persistently.

[0157] Step 12: Use the parameters obtained in steps 8, 9, 10, and 11 and C 1 , G 1 , C 2 , G 2 , C 3 , G 3 , C 4, G 4 Initialize the calculation process of 7.1, 7.2, 7.3, 7.4, and 7.5, and Replace with C 1 , C 2 , C 3 , C 4 , get the value for C 1 , C 2 , C 3 , C 4 The classification model.

[0158] Step 13: For each test data, execute step 6 first, and then send it to the C 1 , C 2 , C 3 , C 4 The classification model assigns the U corresponding to the ICD9 code that has never been seen c 、V c Set to 0 100 , and obtain the classification results of 4 models Will The ICD9 codes are assigned to the corresponding medical records text.

[0159] The preferred embodiments of the present invention are described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

[0160] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0161] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A method for automatically assigning ICD9 codes to medical record text based on label attributes and feature enhancement, characterized in that: include: Step 1, obtain data from the Internet, including medical record text and its corresponding ICD9 code, ICD9 code description text, and ICD9 code hierarchy; remove words that appear less than 2 times and more than 3000 times in medical record text and ICD9 code description text, and process the length of medical record text into N, truncate those greater than a fixed length, and fill in "PAD" strings less than a fixed length; randomly select M medical records from the acquired data as training data sets, and store all the words in the dictionary, where the number of words is V; Step 2: Use the Gensim tool to train Skip-gram word vectors based on all the training set medical records and ICD9 code description texts to obtain the d-dimensional vector representation corresponding to each word, forming a word vector matrix E∈R V×d , take E as the word vector matrix E corresponding to the ICD9 description text C ∈R V×d , the word vector matrix E corresponding to the medical record text D ∈R V×d The initial value of Step 3: According to the hierarchical structure of ICD9 codes, ICD9 codes are divided into 4 groups: 1 , C 2 , C 3 , C 4 ; C 4 The set of ICD9 codes corresponding to the medical record text in all data is obtained Represents C 4 ICD9 code and C 3 The relationship between ICD9 codes; Represents C 4 The i-th ICD9 code belongs to C 3 The jth ICD9 code in the same way is used to obtain I 3 ,I 2 ; Step 4: Get C through the training data set 4 Correspondence Matrix Represents C 4 The j-th ICD9 code in is assigned to the i-th case in the training set; get Step 5: Generate C using the ICD9 code co-occurrence matrix generation algorithm 4 ICD9 code co-occurrence matrix corresponding to the set In the algorithm, ω and f are positive integers, p∈[0,1]; On this basis, Get G 3 , G 2 , G 1 ; Step 6: Represent each medical record text as a matrix consisting of word one-hot vectors When the nth position of the ith document is the vth word otherwise C 4 The descriptive text of each ICD9 code is represented as a V-dimensional vector S c，v The value of is the number of times the vth word appears in the cth ICD9 code description text; The description text corresponding to each ICD9 code in the other set is represented as Where ch(c) represents the direct child node of node c; Step 7: M medical records and their corresponding Construct a training data set, randomly divide the training data set into several batches of size B, and perform parameter training in batches; Step 8: Repeat step 7 for 10-20 rounds to obtain the values ​​of each parameter and the classification results of each medical record text in the last round. And save it persistently; Step 9: Collect the medical records and their corresponding Construct a training data set for C 2 Parameter learning is performed on the ICD9 code in the medical record text; due to the long-tail distribution of medical record text, some ICD9 codes have very little corresponding training data and parameters are difficult to train. 1 The classification results in C 2 The code in the code performs negative sampling; first, Sort from high to low, take the first Z 1 The set C(i) is formed, and the set Among them, ch(C(i)) represents the direct child nodes of all nodes of C(i); then, use G 2 , C 2 , Replace the original G 1 , C 1 , Re-execute steps 7 and 8; when re-executing step 7, set the initial values ​​of each parameter to the values ​​obtained in step 8; after re-executing step 8, new parameters will be obtained. And save it persistently; Step 10: Collect the medical records and their corresponding Construct a training data set for C 3 Parameter learning of ICD9 codes in Sort from high to low, take the first Z 2 The set C(i) is formed, and the set Then, use G 3 , C 3 , Replace the original G 1 , C 1 , Re-execute step 7 and step 8; when re-executing step 7, set the initial value of each parameter to the parameter value obtained in step 9; re-execute step 8 to obtain new parameters, And save it persistently; Step 11: Collect the medical records and their corresponding Construct a training data set for C 4 Parameter learning of ICD9 codes in Sort from high to low, take the first Z 3 The set C(i) is formed, and the set Then, use G 4 , C 4 , Replace the original G 1 , C 1 , Re-execute step 7 and step 8; when re-executing step 7, set the initial value of each parameter to the parameter value obtained in step 10; re-execute step 8 to obtain new parameters, And save it persistently; Step 12: Use the parameters obtained in steps 8, 9, 10, and 11 and C 1 , G 1 , C 2 , G 2 , C 3 , G 3 , C 4 , G 4 Initialize the calculation process in step 7, Replace with C 1 , C 2 , C 3 , C 4 , get the value for C 1 , C 2 , C 3 , C 4 Classification model of Step 13: For each test data, execute step 6 first, and then send it to the C 1 , C 2 , C 3 , C 4 The classification model will classify the U corresponding to the ICD9 code that has never been seen c 、V c Set to 0 d , and obtain the classification results of 4 models Will The ICD9 codes are assigned to the corresponding medical records.

2. The method for automatically assigning ICD9 codes to medical record text based on label attributes and feature enhancement according to claim 1, characterized in that: Step 7 includes: Step 7.1: Use the description text of the ICD9 code as the label attribute of the ICD9 code. 1 For each ICD9 code, Get the label attribute characteristics of each ICD9 code Step 7.2: For C 1 For each ICD9 code in Perform feature extraction to obtain a new representation of label attribute features in, is the parameter matrix; Step 7.3: Target C 1 Each ICD9 code in C 1 The co-occurrence relationship between ICD9 codes, that is, the co-occurrence matrix G 1 , use (2) and (3) to calculate the attribute features Perform feature enhancement to obtain enhanced label attribute features in, is the parameter matrix; Step 7.4: Target C 1 Each ICD9 code in U c Perform adaptive fusion, U c ∈R d is the semantic vector of the cth ICD9 code defined additionally, and its initial value is The fusion calculation process is as follows: (4), (5), (6) and (7), where T W ∈R d×d , A W ∈R 3×3d is the parameter matrix, Represents vector concatenation, q∈R 3d ,q′∈R 3 Indicates the proportion of the three parts; each ICD9 code will determine the fusion ratio according to its own characteristics, and obtain the ICD9 code semantic representation W c ; t(x)=sigmoid(T W ReLU(x)) (4) Step 7.5: Perform parameter training for each medical record in the batch; Step 7.6: Calculate the cross entropy loss corresponding to the batch according to formula (24), where Indicates whether the i-th medical record text in the training set is assigned the c-th ICD9 code; Step 7.7: Use the Adam optimization algorithm to adjust the parameter E according to the loss. C , U c 、T W , A W 、E D ,{CNN l，k } l＝1：L，k＝1：K , T , R T , V c 、T H , A H Learning is performed with a learning rate of μ.

3. The method for automatically assigning ICD9 codes to medical record text based on label attributes and feature enhancement according to claim 1, characterized in that: Step 7.5 includes: Step 7.5.

1. Pass Get the word vector representation of the medical record text content Will The medical record text is sent to a multi-layer convolutional network for feature extraction. The calculation process is as follows: (8) and (9); where CNN lk represents the kth CNN in the lth layer, CNN ：，k The corresponding convolution kernel size is size(k); all CNN l，k The movement amplitude is 1, the number of input channels is d, and the number of output channels is 2*d; pad k ()The calculation process is: If size(k) is not divisible by 2, or Add (size(k)-1) / 2 0 vectors before and after the first dimension of d ; If size(k) is divisible by 2, then or Add (size(k)-1) / 2 zeros to the first dimension of d , add size(k) / 2 0s to the end d ; After each CNN l，k Then we will get the representation of each text L×K CNNs lk All semantic representations of each medical record text will be obtained Step 7.5.2: For the i-th medical record text For each ICD9 code in from Get the closely related content and its semantic representation Will All are set to 0 d , the calculation process is as follows: (10) and (11); where λ T ∈[0, 1], R T ∈R d×d is the parameter matrix, The calculation process of is as follows: (12), (13), (14) and (15); where: represents all semantic vectors of the nth word in the i-th medical record text obtained in step 7.5.1, a i，c，n，： ∈R (L×K) Indicates the closeness between the c-th ICD9 code and the different semantic representations of the n-th word in the ith medical record text; represents the semantic representation of the nth word obtained by comparing the i-th medical record text with the c-th ICD9 code, β i，c，： ∈R N Indicates the closeness between the cth ICD9 code and different words in the i-th medical record text; Step 7.5.3: For each ICD9 code in Perform feature extraction and obtain Step 7.5.4: For each ICD9 code in, according to the co-occurrence relationship between ICD9 codes, that is, the co-occurrence matrix G 1 , using equations (17) and (18) to Perform feature enhancement to obtain Step 7.5.5: For each ICD9 code, additional Extract relevant content, V c ∈R d Extract the vector corresponding to the relevant content of the cth ICD9 code, with the initial value being From another perspective, we can obtain the content and semantic representation related to the cth ICD9 code in the i-th medical record text. The calculation process of is as follows: (12), (13), (14) and (15); Step 7.5.6: For each ICD9 code in Adaptively fuse and obtain the semantic representation H of the final i-th medical record text and the c-th ICD9 code related content i，c ∈R d ; The calculation process is as follows: (19), (20), (21) and (22), where T H ∈R d×d , A H ∈R 3×3d is a parameter matrix, q′ represents the fusion ratio of the three parts, and will have different values ​​for different ICD9 codes; t(x)=sigmoid(T H ReLU(x)) (19) Step 7.5.7: For each ICD9 code in H, use formula (23) to i，c With W c By comparison, the classification result y is obtained i，c ∈[0, 1], y i，c represents the probability that the i-th medical record text is assigned the c-th ICD9 code; par(c) represents the parent node of the c-th ICD9 code, y i，par(c) Indicates that the i-th medical record text is the comparison result of par(c); if par(c)=φ, y i，par(c) =1; y i，c =y i，par(c) sigmoid(W c H i，c T ) (23)。

Images