Disease classification icd automatic coding method and device based on contrast learning

Abstract

The application discloses a disease classification ICD automatic coding method and device based on contrast learning, and belongs to the field of natural language processing and artificial intelligence. The technical problem to be solved by the application is how to use natural language processing technology to match corresponding diagnosis coding, i.e., disease classification coding, for patient electronic medical record texts, so as to reduce the workload of medical personnel, improve the automatic coding matching accuracy, and improve the matching speed. The technical scheme adopted is as follows: ① a disease classification ICD automatic coding method based on contrast learning, which comprises the following steps: S1, constructing an ICD automatic coding model training data set; S2, constructing an ICD automatic coding model; and S3, training the ICD automatic coding model. ② an ICD automatic coding device based on contrast learning, which comprises an ICD automatic coding model training data set construction unit, an ICD automatic coding model construction unit and an ICD automatic coding model training unit.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly designs a disease classification ICD automatic coding method and device based on contrast learning. BACKGROUND

[0002] The ICD automatic coding task is usually performed by specialized medical personnel, who manually assign appropriate ICD codes to electronic medical record texts. However, manual assignment is a labor-intensive task, which not only increases the workload of medical personnel, but also wastes medical resources. In the early 20th century, people began to use statistical methods to solve the ICD automatic coding problem. With the rapid development of the Internet, the traditional medical industry has begun to connect with the Internet, and a large number of applications combined with the medical field have emerged, including ICD automatic coding. ICD automatic coding can reduce the workload of medical staff, and only needs the doctor to input the electronic medical record text into the ICD automatic coding device to obtain the matching ICD code. The ICD automatic coding device not only saves the doctor's time, but also reduces the hospital's labor cost.

[0003] However, the electronic medical record text often includes a large number of medical terms, disease abbreviations, and obscure expressions of doctors, and each electronic medical record text usually involves multiple ICD labels, i.e., ICD codes. Moreover, the electronic medical record text is usually a long character sequence, only a small part of the key text is related to a specific ICD code, and contains a large amount of noise. At the same time, the label space of ICD code is very large, with more than 22000 codes in ICD-9 and more than 170000 codes in the latest ICD-10. The large label space means a serious long-tail problem. So far, there is no method that can completely overcome the above problems and accurately complete the ICD coding task of electronic medical record text in the ICD automatic coding classification task of electronic medical record text.

[0004] In recent years, many neural network models have been applied to the ICD automatic coding task to improve the representation quality of electronic medical record text or utilize the hierarchical relationship between ICD codes to improve the accuracy of the ICD automatic coding task. However, for the ICD automatic coding task, the model needs to learn the representation of different codes for prediction, and learning from electronic medical record text without introducing external knowledge often encounters the long-tail problem. By observing the ICD code description file, it is found that ICD codes with the same higher-level code often have similar text representations. This often causes the model to misjudge when predicting the code.

[0005] How to alleviate the long tail problem is a key to solve the ICD automatic coding task. At present, the ICD automatic coding model does not consider the abbreviation of medical terms in the electronic medical record text, and the case that one disease has multiple expressions, for example, the concept text corresponding to the ICD code 244.9 is "Unspecified hypothyroidism", however, the ICD code may appear in the electronic medical record text as "low t4" or "subthyroidism", the above-mentioned case will make it difficult for the model to accurately identify the key text in the electronic medical record text, thereby causing interference to the ICD automatic coding task. At the same time, the existing model does not consider the similarity of different ICD coding texts, so it will cause the ICD automatic coding model to be difficult to accurately identify medical terms when coding the electronic medical record text, thereby leading to the electronic medical record text being assigned with wrong ICD code. In view of the above problems, the present application proposes an ICD automatic coding method and device for disease classification based on contrast learning. The method first realizes the use of a data preprocessing module to construct an ICD automatic coding model training data set; then a positive and negative example generation and selection module is constructed to generate and select appropriate positive and negative examples for the electronic medical record text in the training set, improve the model's ability to distinguish similar concept texts and enhance the model's ability to identify different expressions of medical terms, and alleviate the long tail problem; a label-by-label contrast learning attention module is constructed, that is, the ICD automatic coding training set and the selected positive and negative examples are input into the embedding layer to learn the corresponding shallow semantic representation, then they are input into the BiLSTM to further capture semantic information representation, and at the same time, contrast learning is used to focus on the mutual relationship between labels and the interaction between labels and features; a prediction module is realized to assign appropriate ICD code to each electronic medical record text. SUMMARY

[0006] In view of the deficiencies of the existing ICD automatic coding method, the present application proposes an ICD automatic coding method and device for disease classification based on contrast learning to solve the problem of ICD automatic coding using natural language processing technology. According to the electronic medical record text of the patient, the corresponding international disease classification ICD code is automatically assigned, so as to reduce the work burden of medical personnel for coding manually and alleviate the problem of medical resource shortage.

[0007] The method and device realize a label-by-label contrast learning attention network structure, which obtains shallow semantic representations of electronic medical record texts and positive and negative sample texts through embedding layers, and then captures semantic information representations of both through BiLSTM; further, a label-by-label attention mechanism is used to map the semantic information representation of the electronic medical record text to the ICD code dimension, and contrast learning is used to focus on the mutual relationship between ICD codes and the interaction between ICD codes and the semantic information representation of the electronic medical record text. The present application proposes a difficult contrast learning structure based on external knowledge, which introduces ICD code synonymous concepts in the UMLS database as positive samples to improve the model's ability to recognize different expressions of medical terms; at the same time, the structure uses the hierarchical relationship of ICD codes to construct a hierarchical structure graph, and uses the concept texts of sibling nodes in the hierarchical structure graph as negative samples, thereby improving the model's ability to distinguish similar concept texts.

[0008] The technical task of the present application is realized in the following manner: a disease classification ICD automatic coding method based on contrast learning, comprising the following steps:

[0009] S1, constructing a data processing module: first, constructing an ICD automatic coding model training data set, processing electronic medical records, and associating the discharge summary part of each patient's electronic medical record text with its ICD code to form a training data; each electronic medical record text corresponds to multiple diseases, i.e. is assigned multiple ICD codes, and finally all training data are aggregated to construct an ICD automatic coding model training data set;

[0010] S2, constructing an ICD automatic coding model: the ICD automatic coding model is composed of a positive and negative sample generation and selection module, a label-by-label contrast learning attention module, and a prediction module;

[0011] S201, constructing a positive and negative sample generation and selection module: the positive and negative sample generation and selection module includes two parts, a positive and negative sample generator and a positive and negative sample selector; the positive and negative sample generator is used to construct a positive and negative sample candidate set, i.e. to assign all positive and negative samples of each ICD code in the ICD automatic coding model training data set in S1 as the positive and negative sample candidate set; the positive and negative sample selector is used to select suitable positive and negative samples from the positive and negative sample candidate set generated by the positive and negative sample generator according to certain rules, so that the model can better perform contrast learning;

[0012] S202, constructing a label-by-label contrast learning attention module: the label-by-label attention module mainly has four parts, which are embedding layer, BiLSTM, label-by-label attention, and difficult contrast learning based on external knowledge; wherein the embedding layer part firstly uses the pre-trained Word2Vec to perform character embedding on the text of the positive and negative examples in S2 and the electronic medical record text in S1, and then sends it into the Embedding layer to obtain its respective shallow semantic representation; then it is sent into BiLSTM to capture the semantic information representation of the two respectively; then further in the label-by-label attention manner, the semantic information representation of the electronic medical record text is mapped to the ICD code dimension to obtain the semantic vector specific to the ICD code, each row of which respectively represents the representation learned by the electronic medical record text for a specific ICD code; in order to better learn the mutual relationship between ICD codes and the interaction between ICD codes and electronic medical record text semantic information, the difficult contrast learning based on external knowledge is introduced, aiming to improve the recognition ability of the model to different expressions of medical terms and the ability to distinguish similar concept texts;

[0013] S203, constructing a prediction module: the prediction module trains a classifier for each ICD code to predict the ICD code; receiving the semantic vector specific to the ICD code learned in the label-by-label contrast learning attention module, passing each row through the sigmoid function and setting a fixed threshold to predict the ICD code;

[0014] S3, training the ICD automatic coding model: combine the two loss functions with certain weights to optimize the model; first, calculate the binary cross-entropy loss function by comparing the prediction in S203 with the true label, and second, calculate the difficult contrast learning loss function based on external knowledge, and then weight the sum of the two according to the weight ratio 1:λ as the final loss function; using this loss function, the ICD automatic coding model constructed in S2 is optimized and trained on the ICD automatic coding model training data set constructed in S1. When the model in this method has not been fully trained, it needs to be trained on the training data set to optimize the model parameters; when the model is trained, the model can predict the corresponding ICD code for the input electronic medical record text.

[0015] An ICD automatic coding device for disease classification based on contrast learning, the device comprising: an ICD automatic coding model training data set construction unit, an ICD coding model construction unit, and an ICD automatic coding model training unit; each unit realizes the functions of steps S1, S2, and S3 in the ICD automatic coding method for disease classification based on contrast learning, and the specific functions of each unit are as follows:

[0016] The ICD automatic coding model training data set construction unit is configured to process the input electronic medical record text, assign a corresponding ICD code to each electronic medical record text, and finally construct an ICD automatic coding model training data set;

[0017] The ICD automatic coding model construction unit comprises two parts, a positive and negative sample generation and selection module, which is configured to construct positive and negative samples for each training sample by using the hierarchical structure of the ICD code corresponding to the training sample and the external knowledge of the ICD code, and a label-by-label contrast learning attention unit and a prediction module, which are configured to construct an embedding layer, a BiLSTM, a label-by-label attention, and a difficult contrast learning based on external knowledge; the label-by-label attention finally maps the encoded text semantic information representation to the ICD code dimension to obtain an ICD code specific semantic vector; the difficult contrast learning based on external knowledge focuses on the mutual relationship between ICD codes and the interaction between ICD codes and electronic medical record text semantic information, thereby better constraining the model and improving the prediction accuracy of the model; and the ICD code specific semantic vector output by the label-by-label attention module is finally sent to the prediction module, a sigmoid function is used for each row, and a fixed threshold is set for prediction.

[0018] The ICD automatic coding model training unit is configured to construct a loss function and an optimization function required in the model training process, and finally complete the training and optimization of the model.

[0019] A storage medium having a plurality of instructions stored therein, the instructions being loaded by a processor to execute the steps of the contrast learning based ICD automatic coding method described above.

[0020] An electronic device comprising the storage medium described above and a processor configured to execute the instructions in the storage medium.

[0021] The contrast learning based disease classification ICD automatic coding method and device have the following advantages:

[0022] (1) The present application uses external knowledge contrast learning to alleviate the long tail problem in the ICD automatic coding task, constructs a hierarchical structure diagram of ICD codes to introduce hierarchical knowledge of ICD codes, and provides a new idea for subsequent research;

[0023] (2) The present application can quickly and efficiently process existing electronic medical record texts, so that the model can quickly learn the specific representation according to the characteristics of different codes;

[0024] (Three), the application uses label-by-label attention mechanism, calculates the attention weight between each position in the input electronic medical record text semantic information representation and a specific ICD code, and thus pays attention to the information related to the ICD code. The important sentences in the electronic medical record text are extracted through the attention weight between the ICD code and the electronic medical record text, and finally the intuitiveness and reliability of the model prediction result are improved;

[0025] (Four), the application effectively provides ICD code candidates for medical coding personnel, reduces the number of manual interventions, improves the efficiency of medical coding, and makes the electronic medical record well applied, which is more conducive to medical data statistics and analysis;

[0026] (Five), the application uses a label-by-label attention module to map the electronic medical record text to the ICD code dimension, and constructs a relationship between each word in the electronic medical record and the ICD code, so as to use as much information in the electronic medical record text as possible, thereby improving the accuracy of the ICD coding task;

[0027] (Six) the application uses natural language processing technology to solve the ICD code matching problem for medical personnel, reduces the workload of doctors, and improves the work efficiency of coding personnel;

[0028] (Seven) the application uses the hierarchical structure knowledge of ICD code, and uses the sibling nodes of ICD code as negative samples. Because the sibling nodes of ICD code often have similar concept texts with the correct ICD code description, the difficult contrast learning based on external knowledge can improve the model's ability to distinguish ICD codes with similar concept texts but different substances, and can help the model to better learn the representation of ICD code;

[0029] (Eight) the application uses difficult contrast learning based on external knowledge, introduces the synonymous concepts of ICD code in the UMLS database as positive samples, thereby improving the model's ability to recognize different expressions of the same medical term. BRIEF DESCRIPTION OF DRAWINGS

[0030] The application will be further described below with reference to the drawings.

[0031] Figure 1 Structure diagram of the disease classification ICD automatic coding method and device based on contrast learning

[0032] Figure 2 Framework diagram of the disease classification ICD automatic coding method and device based on contrast learning Figure 3 Positive and negative sample generation and selection module schematic diagram DETAILED DESCRIPTION

[0033] A disease classification ICD automatic coding method based on contrast learning of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Embodiment 1

[0035] The overall model framework structure of the present application is shown as Figure 1 The main part includes positive and negative example generation and selection module, label-by-label contrast learning attention module and prediction module. Each electronic medical record text and the corresponding ICD code constitutes a training sample; all training samples are collected to form an ICD automatic coding model training data set, which is transmitted to the positive and negative example generation and selection module and the label-by-label contrast learning attention module. The positive and negative example generation and selection module first processes the received ICD automatic coding model training data set to construct positive and negative examples for each training sample for contrast learning; the positive and negative example generation and selection module includes two parts, one is a positive and negative example generator and the other is a positive and negative example selector; the positive and negative example generator is used to construct a positive and negative example candidate set, that is, to assign each ICD code all its positive and negative examples, and to construct the candidate set before training the model to better select positive and negative examples and save memory expenditure. Specifically, as shown in Figure 3As shown, for a training sample, the positive and negative example generator includes two parts of positive example generator and negative example generator; for the positive example generator, according to the concept unique identifier CUI of ICD code, the corresponding synonymous concept is found in the UMLS database, and the above synonymous concept text is composed into a set as a positive example candidate set; for the negative example generator, in order to improve the classification ability of the model to the ICD code, the hierarchical structure knowledge of the ICD code is introduced into the negative example generator, the ICD code is first constructed into a hierarchical structure graph, then the brother nodes corresponding to the ICD code are selected in the hierarchical structure graph, and the concept texts of the above brother nodes are constructed into a set as the negative example candidate set of the ICD code; the positive and negative example selector is used to select the positive and negative example candidate sets generated in the positive and negative example generator, specifically, the positive and negative example selector includes two parts of positive example selector and negative example selector; for the positive example selector, considering that when the ICD code is assigned to a certain electronic medical record text, because the medical terms used by the doctor in recording the patient's condition are random, it can be considered that the probabilities of the corresponding synonymous concepts of the ICD code appearing in the current electronic medical record text are the same, therefore the positive example selector selects all synonymous concept texts of the ICD code as positive examples; for the negative example selector, considering that the brother nodes of the ICD code are also the special cases of correct ICD codes, therefore the concept texts corresponding to the above brother nodes are first excluded in the negative example candidate set, and after the exclusion, the remaining negative example candidate set is randomly selected as negative examples, and the positive and negative examples selected by the positive and negative example selector are transmitted into the label-by-label comparison learning attention module.The label-by-label contrast learning attention module is composed of an embedding layer, a BiLSTM, a label-by-label attention, and a difficult contrast learning based on external knowledge. The embedding layer first uses Word2Vec to perform word embedding on the electronic medical record text in the ICD automatic coding model training data set transmitted by the data processing module, and inputs the character embedding into the Embedding layer as the initialization vector of the Embedding layer, and obtains the shallow semantic representation of the electronic medical record. Then, the BiLSTM is used to encode the above-mentioned electronic medical record shallow semantic representation to capture the semantic information representation of the electronic medical record text. Secondly, the positive and negative samples generated and selected by the positive and negative sample generation and selection module are input into the embedding layer to perform word embedding using Word2Vec, and the character embedding is input into the Embedding layer to obtain the shallow semantic representation of the positive and negative samples. Then, the same BiLSTM is used to encode the shallow semantic representation of the positive and negative samples to capture the semantic information representation of the positive and negative samples. Further, the label-by-label attention mechanism is used to map the semantic information representation of the electronic medical record text into the ICD coding dimension, to learn the specific representation of each type of ICD coding, that is, to obtain the semantic vector specific to the ICD coding. At the same time, the contrast learning is used to focus on the mutual relationship between the ICD coding and the interaction between the ICD coding and the features, and finally the semantic vector specific to the ICD coding learned in the label-by-label contrast learning attention is input into the prediction module. The prediction module uses the sigmoid function to calculate the probability of the occurrence of the ICD coding, and sets a fixed threshold to make the final prediction.

[0036] Embodiment 2:

[0037] The application is an ICD automatic coding method based on contrast learning, and the specific steps are as follows:

[0038] S1, first, an ICD automatic coding model training data set is constructed, and the electronic medical record is processed, and the discharge summary part of each patient's electronic medical record text is associated with its ICD coding to form a training data. If each electronic medical record text corresponds to multiple diseases, the training data will be assigned multiple ICD codes. Finally, all the training data are aggregated to obtain the ICD automatic coding model training data set.

[0039] S101, constructing an electronic medical record text: first, in the electronic medical record database, select the electronic medical record with the attribute of inpatient summary as the final electronic medical record text;

[0040] For example, the electronic medical record text constructed in the MIMIC III data set is as follows:

[0041] Multiple diagnoses exist for the same patient, so the electronic medical record is connected with the patient number HADM ID to obtain all electronic medical record texts of the same patient. The electronic medical record text contains multiple attribute texts, and some attribute texts are not helpful for the ICD code prediction task, so the attribute text "Discharge summary" describing the patient's hospital diagnosis is selected as the electronic medical record text.

[0042]

[0043]

[0044] S102, assigning ICD codes to electronic medical record texts: building an ICD automatic coding training data set requires assigning each electronic medical record text built in S101 to its corresponding ICD code;

[0045] For example, in the MIMIC III data set:

[0046] The diagnoses_icd table provides the hospital ICD code corresponding to the patient number, and the procedures_icd table provides the diagnosis ICD code corresponding to the patient number. When assigning the final ICD code, the hospital ICD code and the diagnosis ICD code are selected to be unioned as the ICD code corresponding to the electronic medical record text.

[0047] In python, the implementation code of steps S101 and S102 in the MIMIC III data set is as follows:

[0048] select_statement="SELECT subject_id,text FROM mimiciii.noteeventsWHERE hadm_id={}and category='Discharge summary'ORDER BY charttime,chartdate,description desc".format(admission_id)

[0049] select_statement="SELECT icd9_code FROM mimiciii.diagnoses_icd WHEREhadm_id={}ORDER BY seq_num".format(admission_id)

[0050] cur.execute(select_statement)

[0051] diag_chapter_labels, diag_three_character_labels, diag_full_labels = process_codes(cur, True, top_n_labels)

[0052] process_codes(cur, True, top_n_labels)

[0053] select_statement = "SELECT icd9_code FROM mimiciii.procedures_icd WHERE hadm_id={}".format(admission_id)

[0054] cur.execute(select_statement)

[0055] proc_chapter_labels, proc_three_character_labels, proc_full_labels = process_codes(cur, False, top_n_labels)

[0056] Where select_statement represents extracting the corresponding electronic medical record text according to the patient number HADM_ID in the MIMIC III dataset. The function of the process_codes function is to divide the ICD code into three parts according to its hierarchical structure, which represents the chapter information of the ICD code, the high-level code of the ICD code, and the ICD code, respectively, chapter_labels, three_character_labels, and full_labels. The above three parts of the ICD code are combined with the electronic medical record text according to the patient number HADM_ID to form a complete training data. Finally, a training data contains the following six parts:

[0057]

[0058] S2, Constructing an ICD automatic coding model: The ICD automatic coding model is composed of a positive and negative example generation and selection module, a label-by-label contrast learning attention module, and a prediction module.

[0059] S201, construct positive and negative example generation and selection module: the positive and negative example generation and selection module includes two parts, one is a positive and negative example generator, and one is a positive and negative example selector; the positive and negative example generator is used to construct the positive and negative example candidate set, that is, to assign all the positive and negative examples of each ICD code in the ICD automatic coding model training data set in S1 as the positive and negative example candidate set; the positive and negative example selector is used to select suitable positive and negative examples from the positive and negative example candidate set generated by the positive and negative example generator according to certain rules, so that the model can better perform contrastive learning; S20101, construct positive and negative example generator: including two parts, one is a positive example generator, and one is a negative example generator, which are used to assign the corresponding positive and negative example candidate set to all ICD codes appearing in S1, and save it in dictionary format;

[0060] S2010101, construct positive example generator: introduce external knowledge UMLS database, and construct the corresponding positive example candidate set for ICD code by means of the synonymous concept relationship of UMLS database;

[0061] For example, in the MIMIC III data set:

[0062] Each ICD code has its corresponding concept unique identifier CUI, and the corresponding synonymous concept is found in the UMLS database according to the CUI, and the [ICD code, synonymous concept text] pair is stored as a json file as a positive example candidate set for subsequent use; the [ICD code, synonymous concept text] pair is as follows:

[0063]

[0064] S2010102, construct negative example generator: use the hierarchical relationship of ICD code to construct the ICD code hierarchical relationship graph; use the concept text of the sibling nodes of ICD code in the hierarchical relationship graph to construct its negative example candidate set;

[0065] For example, in the MIMIC III data set:

[0066] ICD codes have hierarchical relationships, and through hierarchical subdivision, they can often more accurately describe the relevant situation of the disease; the hierarchical relationship of ICD code often has an intuitive connection with its ICD code, when two ICD codes belong to the same high-level category, the two ICD codes are separated by decimal points, and they will have the same number before the decimal point, the concept text corresponding to all sibling nodes of the ICD code is constructed as a set as a negative example candidate set; for example:

[0067]

[0068] S20102, constructing positive and negative example selector: containing two parts, one is positive example selector, one is negative example selector, both of which select the positive and negative example candidate set constructed in S20101 to obtain the final positive and negative examples;

[0069] S2010201, constructing positive example selector: processing each positive example candidate text in the positive example candidate set constructed in S2010101, removing special punctuation marks and splicing all positive example texts corresponding to ICD codes as the final positive examples;

[0070] For example, in the MIMIC III dataset, the implementation is realized by using python, and the code is as follows:

[0071]

[0072] S2010202, constructing negative example selector: data cleaning is performed on the negative example candidate set constructed in S2010102, and negative examples are randomly selected from the negative example candidate set after cleaning;

[0073] For example, in the MIMIC III dataset, the implementation is realized by using python, and the code is as follows:

[0074]

[0075] Finally, the constructed positive and negative example texts and their electronic medical record texts are as follows:

[0076]

[0077]

[0078] S202. Constructing a Label-by-Label Contrast Learning Attention Module: The label-by-label attention module consists of four main parts: an embedding layer, a BiLSTM, label-by-label attention, and difficult contrastive learning based on external knowledge. The embedding layer first uses a pre-trained Word2Vec to embed characters into the text of the positive and negative examples in S2 and the electronic medical record text in S1. These are then fed into the embedding layer to obtain their respective shallow semantic representations. Subsequently, they are fed into a BiLSTM to capture the semantic information representations of both. Then, using label-by-label attention, the semantic information representation of the electronic medical record text is mapped to the ICD encoding dimension, resulting in an ICD-specific semantic vector. Each row represents the representation learned by the electronic medical record text for a specific ICD encoding. To better learn the interrelationships between ICD encodings and the interaction between ICD encodings and the semantic information of the electronic medical record text, difficult contrastive learning based on external knowledge is introduced. This aims to improve the model's ability to recognize different expressions of medical terminology and distinguish similar conceptual texts.

[0079] S20201, Constructing the Embedding Layer: First, use the pre-trained Word2Vec to embed characters into the electronic medical record text in the ICD autoencoding model training dataset constructed in S1, and embed characters into the positive and negative sample texts in S2010201 and S2010202; send the above character embeddings as initialization vectors into Embedding for encoding, thereby obtaining their respective shallow semantic representations;

[0080] For example, in the MIMIC III dataset:

[0081] Suppose that the electronic medical record text D consists of n words, represented as D = w1, w2, ..., w n , will w i∈1:n The characters are embedded using Word2Vec and then used as the initialization vector for the Embedding layer for encoding. This yields the shallow semantic representation of the electronic medical record as follows: Similarly, for positive and negative sample text S, The corresponding shallow semantic representations are obtained as follows: S,

[0082] The Python code for the above functionality is as follows:

[0083]

[0084] S20202, obtaining bidirectional semantic information using BiLSTM: the shallow semantic representation of the electronic medical record text output in S20201 and the positive and negative sample shallow semantic representations are input into a bidirectional LSTM layer to capture the semantic information representations of the two respectively;

[0085] For example, in the MIMIC III dataset:

[0086] The shallow semantic representation D of the electronic medical record text obtained in S20201 is input into the BiLSTM module to learn its context information and semantic relationship as the final semantic information representation H D ; similarly, for the positive and negative sample shallow semantic representations S, input into the same BiLSTM for learning, and finally obtain their semantic information representations as H S 、

[0087] The python code of the above operation is as follows:

[0088]

[0089] S20203, mapping the semantic information representation of the electronic medical record text to the ICD code dimension using label-by-label attention: the goal is to map the semantic information representation H D of the electronic medical record in S20202 to an ICD code-specific semantic vector, each row of which represents the representation learned by the electronic medical record text for a specific ICD code;

[0090] S2020301, calculating the weight matrix specific to the ICD code: taking the semantic information representation H D of the electronic medical record as input, converting it into a |L| dimensional vector |L|, i.e. an ICD code-specific semantic vector as output, where |L| represents the total number of ICD codes appearing in the ICD automatic coding model training data set; the attention weight between each position in the semantic information representation of the electronic medical record text and a specific ICD code is calculated, and the formula is as follows:

[0091] Z = tanh(WH D ) (1)

[0092] A = Softmax(UZ) (2)

[0093] where The above dimension d a is a manually set hyperparameter, u is the hidden layer dimension of the BiLSTM module constructed in step S20202, and because the BiLSTM is used to capture the semantic information representation, the final dimension is 2u, U is a trainable matrix, and its dimension is The matrix Z and the matrix U are multiplied to calculate the weight matrix specific to the ICD code, and finally the dimension of A is as follows: A ∈ R |L|×n ; for the matrix A, the i-th row represents a weight vector for the i-th code in |L|, and therefore the Softmax is applied at the row level to ensure that the sum of the weight values of each row is 1; wherein the Softmax is an activation function that can normalize a numerical vector into a probability distribution vector, and the sum of the probabilities is 1; wherein tanh is an activation function that can change a numerical vector to [-1, 1] and is used to improve the convergence speed of the model;

[0094] The python code implementation of the above operation is as follows:

[0095]

[0096] S2020302, calculating the electronic medical record text semantic information representation H D specific to the ICD code: taking the electronic medical record text semantic information representation H D and the weight matrix A specific to the ICD code obtained in S2020301 as input, multiplying the two to obtain the semantic vector specific to the ICD code as output; the formula is as follows:

[0097] V = H D A T (3)

[0098] wherein the meaning of the matrix V is that the i-th column represents the representation of the i-th code in |L|, and V ∈ R 2u×|L| ;

[0099] S20204, constructing a difficult contrast learning based on external knowledge: constructing a contrast learning loss function, and converging the positive and negative sample semantic information representations constructed in S201 with the electronic medical record text semantic information representation, denoted as and sent into the difficult contrast learning module based on external knowledge; the loss function of the difficult contrast learning based on external knowledge is defined as follows:

[0100]

[0101] wherein H D , H S represent the meanings which have been explained in S20202 and will not be repeated here; represents the semantic information representation corresponding to the i-th negative sample text; represents the semantic information representation corresponding to the i-th negative sample text, and the number of negative samples depends on the number of negative samples selected when training the ICD automatic coding model, which is a hyperparameter; τ represents a temperature hyperparameter; sim(HD H S H is used to calculate the semantic information representation of electronic medical record text. D The semantic information representation H of the positive sample text S The cosine similarity between them is calculated using the following formula:

[0102]

[0103] Similar to the cosine similarity calculation method described above, its formula is:

[0104]

[0105] The meanings of the symbols have already been explained, and will not be repeated here.

[0106] S203, Constructing the prediction module: The semantic vector V specific to ICD encoding obtained in S20203 is applied to the row level with a sigmoid function to obtain the final prediction V. V is converted into a one-hot vector of length |L| by a fixed threshold of 0.5. The one-hot vector is mapped to ICD encoding by the stored label-position dictionary as the final prediction.

[0107] S3. Training the ICD Autoencoding Model: The model is optimized by combining two loss functions with certain weights. First, a binary cross-entropy loss function is calculated between the predicted and true labels in S203. Second, a difficult contrastive learning loss function based on external knowledge is calculated. The two are then weighted and summed according to a weight ratio of 1:λ to obtain the final loss function. Using this loss function, the ICD autoencoding model constructed in S2 is optimized and trained on the training dataset of the ICD autoencoding model constructed in S1. If the model is not fully trained, it needs to be trained on the training dataset to optimize the model parameters. Once the model is trained, it can predict the corresponding ICD code for the input electronic medical record text.

[0108] S301. Calculate the binary cross-entropy loss function: Compare the one-hot vector corresponding to the predicted ICD code output from S203 with the one-hot vector of the actual ICD code to calculate the binary cross-entropy loss, as shown in the following formula:

[0109]

[0110] Where |L| represents the total number of ICD codes appearing in the ICD autoencoder model training dataset. This represents the prediction result of the ICD autocoding model at the j-th position in a one-hot vector of length |L|, belonging to {0,1}, y jThe result of the onehot vector representing the correct ICD code corresponding to the electronic medical record on the jth code belongs to {0, 1}, and theta is a threshold value selected by the model when predicting, which is a hyperparameter;

[0111] S302, constructing a total loss function: including two parts, respectively, the binary cross entropy loss function calculated in S301, the loss function of the difficult contrast learning based on external knowledge calculated in S20204;And add the two according to the weight of 1: Lambda. The specific formula is as follows:

[0112]

[0113] Theta is a threshold value selected by the model when predicting, which is a hyperparameter; Lambda is a hyperparameter artificially set;

[0114] S303, optimizing the model training: using Adamw as the optimization algorithm, the learning rate is set to 0.001, and the optimization training is carried out in the ICD automatic coding model training data set.

[0115] The optimization function and the optimization training process described above are implemented in pytorch as follows:

[0116]

[0117] Where model.parameters() is the parameter of the model, lr is the learning rate, optimiser.zero_grad() is the function of clearing the gradient, loss is the loss obtained after one round of training of the model. Loss.backward() is used to calculate the gradient of all learnable parameters in the model, and optimiser.step() means that the optimizer updates the parameters according to the calculated gradient.

[0118] The model proposed in the application achieves better results than other models on the MimicIII-50 data set, and the comparison of experimental results is shown in the following table:

[0119] model Macro-AUC Micro-AUC Macro-F1 Micro-F1 Precision-P@5 MultiResCNN 89.3 92.04 59.29 66.24 61.56 CSML 87.5 90.9 53.2 61.4 60.9 MSATT-KG 91.4 93.6 63.8 68.4 64.4 The present invention 92.75 94.71 67.44 72.05 67.6

[0120] Example 3:

[0121] As shown in the accompanying Figure 2 Based on the disease classification ICD automatic coding device based on contrast learning of example 2, the device comprises: an ICD automatic coding model training data set construction unit, an ICD coding model construction unit, and an ICD automatic coding model training unit;Respectively realize the functions of steps S1, S2 and S3 in the disease classification ICD automatic coding method based on contrast learning, and the specific functions of each unit are described as follows:

[0122] The ICD automatic coding model training data set construction unit is configured to process the input electronic medical records, assign a corresponding ICD code to each electronic medical record text, and finally construct an ICD automatic coding model training data set.

[0123] The ICD coding model construction unit includes three parts: a positive and negative example generation and selection module, which constructs positive and negative examples for each training sample for comparative learning by using the hierarchical structure of the correct ICD code contained in the training sample and external knowledge corresponding to the ICD code; a label-by-label comparative learning attention module, which is used to construct an embedding layer, BiLSTM, label-by-label attention, and difficult comparative learning based on external knowledge; the label-by-label attention finally maps the semantic information of the coded electronic medical record text to the ICD code dimension for prediction; the difficult comparative learning based on external knowledge learns the mutual relationship between ICD codes and the interaction between ICD codes and electronic medical record semantic information, thereby better constraining the model and improving the accuracy of model prediction; and a prediction module, which trains a classifier for each ICD code for ICD code prediction.

[0124] The ICD automatic coding model training unit is used to construct the loss function and optimization function required in the model training process, and finally complete the training and optimization of the model. When the model is not fully trained, the training data set needs to be trained to optimize the model parameters; when the model is trained, the model can predict the corresponding ICD code for the input electronic medical record text.

[0125] Embodiment 4:

[0126] The storage medium based on embodiment 2, wherein a plurality of instructions are stored, the instructions are loaded by a processor, and the steps of the ICD automatic coding method based on comparative learning of embodiment 2 are executed.

[0127] Embodiment 5:

[0128] The electronic device based on embodiment 4, the electronic device comprising: the storage medium of embodiment 4; and a processor configured to execute the instructions in the storage medium of embodiment 4.

[0129] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An automatic coding method for disease classification ICD based on contrastive learning, characterized in that, The method is to build a data preprocessing module and train an ICD automatic coding model composed of a positive and negative example generation and selection module, a label-by-label contrast learning attention module, and a prediction module, to focus on the mutual relationship between ICD codes and the interaction between ICD codes and the semantic information representation of electronic medical record texts, and to improve the model's ability to recognize different expressions of medical terms and its ability to distinguish similar concept texts, so as to more accurately assign ICD codes to electronic medical record texts, as follows: S1, build an ICD automatic coding model training dataset: associate the discharge summary part of the patient's electronic medical record text with its ICD code to form an ICD automatic coding training dataset; S2, build an ICD automatic coding model: composed of S201 positive and negative example generation and selection module, S202 label-by-label contrast learning attention module, and S203 prediction module; S3, train the ICD automatic coding model: train the ICD automatic coding model built in S2 on the ICD automatic coding training dataset built in S1; The ICD automatic coding model is trained and optimized as follows: Combine the two loss functions with a certain weight to optimize the model; first, calculate the binary cross-entropy loss function by comparing the prediction in S203 with the true label, and second, calculate the difficult contrast learning loss function based on external knowledge, and then sum them up according to the weight ratio 1:lambda as the final loss function; use this loss function to optimize and train the ICD automatic coding model built in S2 on the ICD automatic coding model training dataset built in S1; when the model is not fully trained, it needs to be trained on the training dataset to optimize the model parameters; when the model is trained, the model can predict the corresponding ICD code for the input electronic medical record text; S301, calculate the binary cross-entropy loss function: calculate the binary cross-entropy loss between the onehot vector of the predicted ICD code with length |L| output by S203 and the onehot vector of the correct ICD code corresponding to the electronic medical record, as follows: where |L| represents the total number of ICD codes appeared in the ICD auto-encoding model training dataset, represents the result predicted by the ICD auto-encoding model for the jth position in the onehot vector of length |L|, belongs to {0, 1}, y j represents the result of the onehot vector of the correct ICD code corresponding to the electronic medical record on the jth code, belongs to {0, 1}, θ is the threshold value selected by the model when predicting, is a hyperparameter; S302, build the total loss function: contains two parts, the binary cross-entropy loss function calculated in S301 and the loss of difficult contrast learning based on external knowledge obtained in S202; and add them according to the weight ratio 1:lambda, as follows: theta is the threshold value selected by the model when predicting, which is a hyperparameter; lambda is a hyperparameter set by humans; (3) Optimize the model training: use Adamw as the optimization algorithm, set the learning rate to 0.001, and optimize the training in the ICD automatic coding model training dataset. 2.The contrastive learning based disease classification ICD automatic coding method of claim 1, wherein Build a data processing module, as follows: S1, build an ICD automatic coding model training data set, process the electronic medical record, associate the discharge summary part of the electronic medical record text of each patient with its ICD code together to form a training data; if a piece of electronic medical record corresponds to multiple diseases, the training data will be assigned multiple ICD codes; finally, all the training data is aggregated to obtain the ICD automatic coding model training data set, as follows: S101, build an electronic medical record text: in the electronic medical record database, select the electronic medical record with the attribute of discharge summary as the source of electronic medical record data; S102, assign ICD codes to electronic medical record texts: according to the electronic medical record data, associate the discharge summary part of the electronic medical record text of each patient with its ICD code together to form a training data; if a piece of electronic medical record corresponds to multiple diseases, the training data will be assigned multiple ICD codes; all the training data is aggregated to obtain the ICD automatic coding model training data set. 3.The contrastive learning based disease classification ICD automatic coding method of claim 1, wherein, Build a positive and negative example generation and selection module, the specific implementation process is as follows: S201, build a positive and negative example generation and selection module: the positive and negative example generation and selection module contains two parts, one is a positive and negative example generator, and the other is a positive and negative example selector; the positive and negative example generator is used to construct a positive and negative example candidate set, that is, to assign all the positive and negative examples of each ICD code in the ICD automatic coding model training data set in S1 as the positive and negative example candidate set; the positive and negative example selector is used to select suitable positive and negative examples from the positive and negative example candidate set generated by the positive and negative example generator according to certain rules, so that the model can better perform contrast learning; S20101, build a positive and negative example generator: build a positive and negative example candidate set, that is, assign all the positive and negative examples of each ICD code in the ICD automatic coding model training data set in S102, and save it as a dictionary format as a positive and negative example candidate set, as follows: S2010101, build a positive example generator: introduce an external knowledge UMLS database, and use the unified concept relationship of the UMLS database to build the corresponding positive example candidate set for the ICD code; S2010102, build a negative example generator: use the hierarchical relationship of ICD codes to build an ICD code hierarchical relationship graph; use the concept text of the sibling nodes of the ICD code in the hierarchical relationship graph to build a negative example candidate set; S20102, build a positive and negative example selector: select the positive and negative example candidate set constructed in the positive and negative example generator in S20101, and obtain the final positive and negative examples, as follows: S2010201, build a positive example selector: process the positive example candidate set generated by the positive example generator in S2010101, first process each positive example candidate text in the positive example candidate set, remove special punctuation marks, and concatenate all the positive example texts corresponding to the ICD code as the final positive example; S2010202, Constructing negative example selector: data cleaning is performed on the negative example candidate set generated by the negative example generator constructed in S2010102, and random selection of negative examples is performed in the cleaned negative example candidate set. 4.The contrastive learning based disease classification ICD automatic coding method of claim 1, wherein Constructing label-by-label contrastive learning attention module S202, Constructing label-by-label contrastive learning attention module: The label-by-label attention module has four parts, which are embedding layer, BiLSTM, label-by-label attention, and difficult contrastive learning based on external knowledge; The embedding layer part first uses the pre-trained Word2Vec to perform character embedding on the text of the positive and negative examples in S2 and the electronic medical record text in S1, and then sends it into the Embedding layer to obtain its respective shallow semantic representation; Then send them into BiLSTM to capture the semantic information representation of both; Then, in a label-by-label attention manner, the semantic information representation of the electronic medical record text is mapped to the ICD code dimension to obtain the semantic vector specific to the ICD code, each row of which represents the representation learned by the electronic medical record text for a specific ICD code; In order to better learn the mutual relationship between ICD codes and the interaction between ICD codes and electronic medical record text semantic information, difficult contrastive learning based on external knowledge is introduced to improve the model's ability to recognize different expressions of medical terms and distinguish similar concept texts; S20201, Constructing embedding layer: first, use the pre-trained Word2Vec to perform character embedding on the electronic medical record text in the ICD auto-encoding model training data set constructed in S1, and perform character embedding on the positive and negative example texts in S2010201 and S2010202; The above character embedding is used as an initialization vector and sent to Embedding for encoding to obtain the respective shallow semantic representation, as follows: Assume that the electronic medical record text D is composed of n words, denoted as D = w1, w2,..., wn. n , w2,..., wn are sent into the Word2Vec for character embedding and are sent into the Embedding layer as the initialization vector of the Embedding layer for encoding to obtain the shallow semantic representation of the electronic medical record as follows: i∈1:n , w2,..., wn are sent into the Word2Vec for character embedding and are sent into the Embedding layer as the initialization vector of the Embedding layer for encoding to obtain the shallow semantic representation of the electronic medical record as follows: Similarly, for the positive and negative samples S, , respectively, the corresponding shallow semantic representations are as follows: S20202, Obtain bidirectional semantic information using BiLSTM: Send the shallow semantic representation of the electronic medical record text and the shallow semantic representation of the positive and negative examples output in S20201 into the bidirectional LSTM layer to capture the semantic information representation of both, as follows: The shallow semantic representation D of the electronic medical record is sent into a BiLSTM module to learn the context information and semantic relationship thereof as a final semantic information representation H D ; similarly, for the positive and negative sample shallow semantic representations S, , are sent into the same BiLSTM for learning, and finally the semantic information representations thereof are H S 、 S20203, mapping the semantic information representation of the electronic medical record text to the ICD coding dimension using label-wise attention: the goal is to map the semantic information representation H of the electronic medical record in S20202 to a semantic vector specific to the ICD coding, each row of which represents a representation learned by the electronic medical record text for a specific ICD coding, respectively. D mapping as a semantic vector specific to the ICD coding, each row of which represents a representation learned by the electronic medical record text for a specific ICD coding, respectively. S2020301, computing a weight matrix specific to ICD codes: representing semantic information of the electronic medical record H D As input, converting to a |L| dimensional vector |L| i.e. semantic vector specific to ICD codes as output, where |L| represents the total number of ICD codes that appear in the ICD auto-encoding model training dataset; computing the attention weight between each position in the semantic information representation of the electronic medical record text and a specific ICD code, formula as follows: Z = tanh(WH D ) A=Softmax(UZ) in In the above dimensions d a is a manually set hyperparameter, where u is the hidden layer dimension of the BiLSTM module constructed in step S20202. Because it uses BiLSTM to capture semantic information representation, the final dimension is 2u. U is a trainable matrix with dimension . Multiply matrices Z and U to calculate the ICD-specific weight matrix. The final dimension of A is as follows: A∈R |L|×n For matrix A, the i-th row represents the weight vector of the i-th encoding in |L|. Therefore, Softmax is applied at the row level to ensure that the sum of the weights in each row is 1. Softmax is an activation function that normalizes a numerical vector into a probability distribution vector, and the sum of the probabilities is 1. tanh is an activation function that transforms a numerical vector into [-1,1], and is used to improve the convergence speed of the model. S2020302, compute the electronic medical record text semantic information representation H D ICD code-specific semantic vector: multiply the electronic medical record text semantic information representation H D and the weight matrix A specific to ICD code obtained in S2020301 as input, and output the ICD code-specific semantic vector; the formula is as follows: V = H D A T where the matrix V has the meaning that the i-th column represents the i-th encoded representation in |L|, V e R 2u×|L| ; S20204, construct difficult contrast learning based on external knowledge: construct a contrast learning loss function, and converge the positive and negative example semantic information representation constructed in S201 with the electronic medical record text semantic information representation, represented as and sent into the difficult contrast learning based on external knowledge module; the loss function of the difficult contrast learning based on external knowledge is defined as follows: Where H D H S The meaning has already been explained in S20202, and will not be repeated here; represents the semantic information representation corresponding to the i-th negative sample text. The number of negative samples depends on the number of negative samples selected when training the ICD autoencoder model, and is a hyperparameter; τ represents the temperature hyperparameter; sim(H D H S H is used to calculate the semantic information representation of electronic medical record text. D The semantic information representation H of the positive sample text S The cosine similarity between them is calculated using the following formula: Similar to the cosine similarity calculation described above, the formula is: Where the meaning of the symbol has been explained before and will not be repeated here. 5.The contrastive learning based disease classification ICD automatic coding method of claim 1, wherein, Constructing prediction module, as follows: S203, Constructing prediction module: The prediction module trains a classifier for each ICD code to predict the ICD code; Receive the semantic vector specific to the ICD code learned in the label-by-label contrastive learning attention module, pass each row through the sigmoid function and set a fixed threshold to predict the ICD code; Apply sigmoid to the ICD code-specific semantic vector V obtained in S20203 at the row level to obtain the final prediction V, convert V to a onehot vector of length |L| by setting a fixed threshold of 0.5, and map the above onehot vector to the ICD code as the final prediction by the stored label-position dictionary.

6. A contrastive learning based disease classification ICD automatic coding apparatus for implementing the contrastive learning based disease classification ICD automatic coding method of any one of claims 1 to 5, characterized in that, The device comprises, The ICD automatic coding model training data set construction unit is configured to preprocess an original data set to construct training data. The ICD coding model construction unit is configured to construct an ICD coding model, which includes two parts. First, a positive and negative example generation and selection module is configured to construct positive and negative examples for each training sample for comparative learning by using the hierarchical structure of the ICD code corresponding to the training sample and the external knowledge of the ICD code. Second, a label-by-label comparative learning attention unit and a prediction module are configured to construct an embedding layer, BiLSTM, label-by-label attention, and difficult comparative learning based on external knowledge. Finally, the semantic vector specific to the ICD code output by the label-by-label attention module is input into the prediction module, sigmoid is applied at the row level, and a fixed threshold is set for prediction. The ICD automatic coding model training unit is configured to construct a loss function and an optimization function required in the model training process, and finally complete the training and optimization of the model. 7.The contrastive learning based disease classification ICD automatic coding apparatus of claim 6, wherein, The model training unit includes, a loss function construction module configured to calculate the error between the prediction result and the true data using a binary cross-entropy loss function and calculate the error between the sample and the positive and negative examples using a difficult comparative learning loss function based on external knowledge; an optimization function construction module configured to train and adjust the parameters in the model training to reduce the prediction error.

8. An electronic device, comprising: It includes: a memory and at least one processor; wherein the memory has stored thereon a computer program; the at least one processor executes the computer program stored in the memory, so that the at least one processor executes the disease classification ICD automatic coding method based on comparative learning according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored therein a computer program, which can be executed by a processor to implement the disease classification ICD automatic coding method based on comparative learning according to any one of claims 1 to 5.

Images