Medical code distribution method based on hierarchical association and diversity enhancement

Abstract

A medical code allocation method based on hierarchical association and diversity enhancement comprises the following steps: S1, data preprocessing and graph construction: based on a tree hierarchical classification architecture of ICD codes, adopting a hierarchical graph model to perform visual representation on the tree hierarchical classification architecture, and constructing a co-occurrence matrix of the clinical record text and the ICD codes for the clinical record text; s2, applying a Graph-BERT graph neural network structure to perform graph structure conversion on a tree structure of an ICD code system, and learning based on an encoder structure of a graph transformer to obtain embedded representation of ICD codes; s3, performing word segmentation processing on a clinical text, inputting the processed clinical text into a bioBERT model to obtain a feature vector of the clinical text, and unifying semantic features of clinical records, hierarchical features of ICD codes and related association information between the semantic features and the hierarchical features through a multi-modal fusion mode; s4, after the final feature vector of the clinical record text is obtained, correlation enhancement prediction is performed by using a CorNet network, so that a related probability matrix is generated, a binary cross entropy loss function is expanded, the binary cross entropy loss function, hierarchical diversity loss and semantic diversity loss jointly form a diversified loss function, and the diversified loss function and a classic model are subjected to comparative analysis, so that the final feature vector of the clinical record text is obtained. Therefore, the accuracy and effectiveness of the method are verified.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of natural language processing and artificial intelligence bioinformatics, and specifically relates to a medical code allocation technology based on hierarchical association and diversity enhancement. BACKGROUND

[0002] Medical code allocation refers to the use of the International Classification of Diseases (ICD) standard system to systematically classify and identify unstructured text information generated in clinical diagnosis and treatment activities. The main goal of this standardized processing is to promote the standardized storage of medical information and realize data intercommunication between different medical information systems. ICD is a disease coding system developed by the World Health Organization (WHO), and the purpose of medical code allocation is to realize the corresponding association between clinical records and standardized ICD codes. At present, ICD coding is mainly based on manual methods, which is laborious and has low accuracy. Therefore, it is very important to develop a computational method for associating clinical records with their corresponding ICD codes.

[0003] Over the past few decades, many computational methods have been proposed, mainly divided into three types: rule-based methods, machine learning-based methods, and deep learning-based methods. Among them, with the application of deep learning in ICD automatic coding, most studies consider combining CNN, RNN, LSTM and their variants to achieve more accurate medical text code allocation. In 2018, a multi-scale convolutional neural network structure was constructed to obtain text patterns of various lengths in clinical text, and residual connections were used to enhance feature transmission and fusion. In order to avoid the problems of gradient disappearance and network structure degradation, residual connections are used between each convolutional block, and the feature information of shallow and deep layers is also considered. In 2021, a model based on the combination of interactive shared representation network (ISRN) and self-distillation mechanism (SDM) was proposed, which uses the output of multiple pre-trained language models (PLM) to construct a shared document representation, and then uses an interactive attention mechanism to weight and fuse the document representation, thereby improving the generalization ability and robustness of the model; In 2024, a BERT model combined with a graph convolutional network (GCN) was proposed for ICD automatic coding. First, the text is encoded using a transformer, then the hierarchical structure of the ICD code is modeled using a GCN, and finally a fully connected layer is used for classification.

[0004] Although various clinical record coding prediction models have been constructed in existing literature, most studies only focus on single feature extraction of clinical text or ICD codes, neither fully considering the influence of the interaction between the two on automatic coding effect, nor lacking coding representation optimization schemes for the long-tail distribution problem. The above problems inspired the key idea of the invention. The present application proposes a new method for medical code allocation based on hierarchical association and diversity enhancement. SUMMARY

[0005] The application aims at a medical code allocation method based on hierarchical association and diversity enhancement.

[0006] The application is a medical code allocation method based on hierarchical association and diversity enhancement, comprising the following steps: S1, data preprocessing and graph construction, based on the tree-like hierarchical classification architecture of ICD code, adopting hierarchical graph model to visually represent it, and constructing the co-occurrence matrix of clinical record text and ICD code; S2, applying the Graph-BERT graph neural network structure to transform the tree structure of the ICD code system into a graph structure, and learning the embedding representation of the ICD code based on the encoder structure of the graph transformer; S3, after the clinical text is segmented and input into the bioBERT model, the feature vector is obtained, and the semantic features of the clinical record, the hierarchical features of the ICD code and the relevant association information between them are unified through multi-modal fusion; S4, after obtaining the final feature vector of the clinical record text, the CorNet network is used for correlation enhancement prediction to generate a relevant probability matrix, and the binary cross-entropy loss function is extended to form a diversified loss function together with the hierarchical diversity loss and semantic diversity loss, and compared with the classical model to verify the accuracy and effectiveness of the above method.

[0007] Compared with the prior art, the application has the following beneficial effects: 1. Higher model accuracy: the method realizes the improvement of medical text coding accuracy and obtains a higher MiF score. MiF score can be counted for each predicted answer in the whole sample, thereby providing a good indication of the performance of the model in most cases, calculated as: Where P is the accuracy of the sample, representing the percentage of all correct prediction results of the model, and the calculation formula is And R is the regression value of the sample, indicating how many of all standard label answers are correctly identified by the model, and the calculation formula is Accuracy represents the confidence level of the answer given by the model, while regression rate represents the fitting degree of the model to the whole problem.

[0008] 2. More balanced prediction ability of different frequency ICD codes: the method realizes the improvement of the accuracy of low-frequency ICD codes, and in the experimental verification of the first 50 high-frequency ICD codes and the last 50 low-frequency codes, the higher the area under the curve (AUC) produced by ROC accounts for the total possible area, the better the performance of the classifier at this frequency. 3. More accurate prediction of the first n results: The method was studied on the public data sets MIMIC-III and MIMIC-IV data sets, and the accuracy of the first n results in the matching results was detected. Generally speaking, the accuracy of the first few results is more important for the model. If the first few results are not related to the clinical record, even if all the results that follow are related, then this model is not satisfactory. The results show that the detection results of the method compared with other models are the best. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, a brief introduction will be given below to the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0010] Figure 1 is a medical code allocation method flowchart based on hierarchical association and diversity enhancement. Figure 2 is a schematic diagram of the ICD code hierarchical association tree. Tables 1 and 2 are comparison results of the method of the present application and 8 kinds of classical correlation methods. DETAILED DESCRIPTION

[0011] As shown in Figure 1 , the present application is a medical code allocation method based on hierarchical association and diversity enhancement. The method first constructs a hierarchical graph for ICD codes for pre-training, then combines the feature information of the clinical record itself and the association information between the entities contained therein and the ICD codes, fuses the features of the two different modalities through a multi-modal attention alignment mechanism, then considers the hierarchical diversity loss in the ICD code and its own semantic diversity loss with the clinical record, calculates the distribution distance of the ICD code in the tree and their semantic similarity to the clinical text, finally adds a CorNet block to the label prediction layer to learn the label correlation, and enhances the original label prediction by constructing dynamic and static correlation matrices. The method includes data preprocessing and graph construction phase, Graph-BERT pre-training ICD code hierarchical graph phase, clinical text encoding and multi-modal feature fusion phase, and CorNet multi-label classification optimization phase, and the steps are: S1, data preprocessing and graph construction, based on the tree-like hierarchical classification architecture of ICD codes, using a hierarchical graph model to visualize the representation, and constructing a co-occurrence matrix of the clinical record text and the ICD code; S2, apply the Graph-BERT graph neural network structure to transform the tree structure of the ICD code system into a graph structure, and learn the embedding representation of the ICD code based on the encoder structure of the graph transformer. S3, after the clinical text is processed by word segmentation, the clinical text is input into a bioBERT model to obtain a feature vector, and a multi-modal fusion mode is used to unify the semantic features of the clinical record, the hierarchical features of the ICD code and the relevant association information between the two; S4, after obtaining the final feature vector of the clinical record text, a CorNet network is used for correlation enhancement prediction to generate a relevant probability matrix, and a binary cross-entropy loss function is extended to form a diversification loss function together with a hierarchical diversity loss and a semantic diversity loss, and a comparison analysis is performed with a classical model to verify the accuracy and effectiveness of the above method.

[0012] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0013] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific test methods.

[0014] As Figure 1 The embodiments of the present application provide a medical code allocation method based on hierarchical association and diversity enhancement, comprising: S1, data preprocessing and graph construction, based on the tree-like hierarchical classification architecture of ICD code, a hierarchical graph model is used for visual representation, and a co-occurrence matrix of clinical record text and ICD code is constructed; Specifically, in the embodiments of the present application, step S1 comprises the following steps: S11, establishment of ICD code hierarchical graph: assuming that Γ is an ICD code hierarchical graph, its vertex set is composed of ICD codes , is the total number of ICD codes, if the ICD codes are in parent-child relationship, the edge weight between them is 1; if are in mutual exclusion relationship, the edge weight is -1, and the adjacency matrix of the ICD code hierarchical graph is obtained according to the weight relationship , wherein the element: ; S12, construction of clinical record and ICD code co-occurrence matrix: the co-occurrence matrix of clinical record text and ICD code is constructed, wherein the rows and columns represent the words in the corpus, and each element in the matrix represents the number of times of co-occurrence of two words in the text, that is, the co-occurrence frequency Then, the co-occurrence matrix is normalized as , and the following is obtained , wherein is the total number of words in the corpus, S2, applying a Graph-BERT graph neural network structure to convert the tree structure of the ICD code system into a graph structure, and learning the embedding representation of the ICD code based on the encoder structure of the graph transformer; Specifically, in the embodiment of the application, step S2 includes the following steps: S21, obtaining the co-occurrence matrix of the clinical records and the ICD codes Then, the tree structure of the original ICD code system is converted into a graph structure by the Graph-BERT model, an ICD hierarchical relationship graph is constructed, and a corresponding adjacency matrix is generated , wherein is the total number of words in the corpus, is the total number of ICD codes; Meanwhile, the text description of the ICD code is feature extracted to convert the semantic information into a node feature vector , wherein corresponds to the length of the description text; S22, learning the embedding representation of the ICD code by the encoder structure based on the graph transformer through the multi-head attention mechanism , and the dimension is Then, the decoder based on the graph transformer is designed to complete two pre-training tasks, including node attribute reconstruction and graph topology reconstruction, and the corresponding loss function values are calculated as follows: ; S3, inputting the clinical text after the tokenization processing to the bioBERT model to obtain the feature vector, and achieving the unification of the semantic features of the clinical records, the hierarchical features of the ICD codes, and the relevant association information between them through the multi-modal fusion method; Specifically, in the embodiment of the application, step S3 includes the following steps: S31, after the tokenization processing of the clinical text, specific marker symbols ([CLS] and [SEP]) are inserted at the start and end positions of each record respectively to identify the sentence boundary and classification information, and a clinical record of tokens is formed ; S32, input the processed text into the bioBERT model, wherein a multi-layer bidirectional transformer encoder structure is adopted, each encoder unit comprises a multi-head self-attention mechanism and a feedforward neural network module, and effective connection between the modules is realized through a residual connection and a layer normalization technology; S33, the label hidden state vector output by the bioBERT model has deep semantic feature and grammatical structure information representing the text, that is, is the vector length of each clinical record, S34, the semantic features of the clinical records, the hierarchical features of the ICD codes and the relevant association information between the two are unified in representation through a multi-modal fusion manner: , wherein, is a bias weight of the co-occurrence matrix, which can be used to control the influence strength of the relevance of the clinical text and the ICD code in the entire coding process, S35, based on the idea of gate, combining multiple information sources and optimizing the final target function, which is used to select which part of the input is more likely to correctly generate the required output, using a multiplication gate that assigns importance to various features, creating a rich multi-modal representation that does not need to be manually adjusted but is learned directly from training data, and finally obtaining the feature vector after gate fusion: ; S4, after obtaining the final feature vector of the clinical record text, the CorNet network is used for correlation enhancement prediction, so as to generate a relevant probability matrix, and the binary cross-entropy loss function is extended, and the hierarchical diversity loss and the semantic diversity loss are jointly constituted into a diversity loss function, which is compared with a classical model to verify the accuracy and effectiveness of the above method.

[0015] Specifically, in the embodiment of the application, step S4 comprises the following steps: S41, since the model is guided in the training of the model by the diversity loss function, after obtaining the final feature vector of the clinical record text, the CorNet network is used for correlation enhancement prediction, so as to generate a relevant probability matrix: , wherein, is a label matrix of the training set, is the original prediction probability without enhancement by the CorNet network; S42, in the calculation part of the loss function, firstly, the most common binary cross-entropy loss function is obtained ; ​S43, input the prediction probability matrix obtained in the CorNet network , by selecting the Top-K probability corresponding ICD code, calculating the distribution distance of these codes in the ICD tree, if the predicted ICD code is concentrated in the same subtree, the loss value increases, forcing the model to disperse the prediction to different subtrees, and obtaining the hierarchical diversity loss , Wherein , that is, the shortest path length of the model in the ICD code hierarchical diagram to , if the predicted ICD code is concentrated in the same subtree, the loss value will increase, so as to force the model to disperse the prediction to different subtrees; S44, if a plurality of high-frequency ICD code semantics are highly similar, the loss value of the high-frequency ICD code needs to be increased, and the model is forced to distinguish similar codes, and the prediction probability matrix is input and text embedding , calculating the semantic similarity of high-frequency ICD code, obtaining the semantic diversity loss function ; S45, the total loss of the model is determined by the loss functions: .

[0016] Table 1 is a comparison of data on the MIMIC-III full data set using the method of the present application and eight automatic medical code allocation methods: Table 1: Model MiF1 MaF1 MiAUC MaAUC p@8 P@k CAML 0.539 0.088 0.986 0.895 0.709 0.561 HyperCore 0.551 0.090 0.989 0.930 0.722 0.579 JAN 0.553 0.091 0.989 0.945 0.735 0.584 LAAT 0.575 0.099 0.988 0.919 0.738 0.591 MSMN 0.584 0.103 0.992 0.950 0.752 0.599 KEPTLongformer 0.599 0.118 - - 0.771 - TF-GCN 0.589 0.109 0.989 0.934 0.758 - PLM-LLM 0.613 0.103 0.988 0.952 0.779 0.615 HCDEM 0.613 0.113 0.989 0.955 0.780 0.621 Table 2 is a comparison of data on the MIMIC-III top 50 highest frequency ICD code data set using the method of the present application and eight automatic medical code allocation methods: Table 2: Model MiF1 MaF1 MiAUC MaAUC p@8 CAML 0.614 0.532 0.909 0.875 0.609 HyperCore 0.663 0.609 0.929 0.895 0.632 JAN 0.671 0.612 0.898 0.931 0.682 LAAT 0.715 0.666 0.946 0.925 0.675 MSMN 0.725 0.683 0.947 0.928 0.680 KEPTLongformer 0.732 0.687 0.947 0.927 0.672 TF-GCN 0.719 0.671 0.953 0.928 0.682 PLM-LLM 0.729 0.670 0.949 0.926 0.704 HCDEM 0.735±0.01 0.691±0.02 0.957±0.02 0.933±0.01 0.705±0.01 Each embodiment of the present application is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0017] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above embodiments. Any technical solution falling within the concept of the present application belongs to the protection scope of the present application. It should be noted that, for ordinary skilled in the art, some improvements and refinements without departing from the principles of the present application should be considered as the protection scope of the present application.

Claims

1. A medical code assignment method based on hierarchical association and diversity enhancement. First, a hierarchical graph is constructed for ICD codes for pre-training. Second, the feature information of the clinical record itself and the association information between the entities contained therein and the ICD codes are combined through a multimodal attention alignment mechanism to fuse the features of these two different modalities. The hierarchical diversity loss in the ICD code and the semantic diversity loss between the ICD code and the clinical record are taken into account. The distribution distance of the ICD code in the tree and their semantic similarity to the clinical text are calculated. Finally, a CorNet block is added to the label prediction layer to learn label correlation. The original label prediction is enhanced by constructing dynamic and static correlation matrices. The method includes a data preprocessing and graph construction stage, a Graph-BERT pre-training ICD code hierarchical graph stage, a clinical text encoding and multimodal feature fusion stage, and a CorNet multi-label classification optimization stage. The steps are as follows: S1. Data preprocessing and graph construction: Based on the tree-like hierarchical classification architecture of ICD codes, a hierarchical graph model is used to visualize them, and a co-occurrence matrix between clinical record texts and ICD codes is constructed; S2. Apply the Graph-BERT graph neural network structure to transform the tree structure of the ICD code system into a graph structure, and learn the encoder structure based on the graph transformer to obtain the embedded representation of the ICD code; S3. After word segmentation, the clinical text is input into the bioBERT model to obtain its feature vector. Multimodal fusion is then used to unify the semantic features of the clinical record and the hierarchical features of the ICD code, as well as the relevant association information between the two. S4. After obtaining the final feature vector of the clinical record text, the CorNet network is used to perform correlation enhancement prediction to generate a relevant probability matrix. The binary cross entropy loss function is expanded to form a diversified loss function together with the hierarchical diversity loss and semantic diversity loss. A comparative analysis is performed with the classic model to verify the accuracy and effectiveness of the above method.

2. The medical code assignment method based on hierarchical association and diversity enhancement according to claim 1 is characterized in that: The specific sub-steps of step S2 are as follows: S21. Co-occurrence matrix of clinical records and ICD codes Finally, the Graph-BERT model is used to transform the tree structure of the original ICD code system into a graph structure, construct an ICD hierarchical relationship graph and generate the corresponding adjacency matrix. ,in is the total number of words in the corpus, is the total number of ICD codes; At the same time, feature extraction is performed on the text description of the ICD code, and the semantic information is converted into a node feature vector ,in Corresponds to the length of the description text; S22, using a graph transformer-based encoder structure, and learning the embedded representation of ICD encoding through a multi-head attention mechanism , whose dimensions are , and then designed a decoder based on the graph transformer to complete two pre-training tasks, including node attribute reconstruction and graph topology reconstruction, and calculated the corresponding loss function values ​​as follows: 。 3. The medical code assignment method based on hierarchical association and diversity enhancement according to claim 1 is characterized in that: The specific sub-steps of step S3 are as follows: S31. After word segmentation of clinical text, insert specific markers ([CLS] and [SEP]) at the beginning and end of each record to identify sentence boundaries and classification information. Clinical records of tokens ; S32. Input the processed text into the bioBERT model, which uses a multi-layer bidirectional transformer encoder structure. Each encoder unit contains a multi-head self-attention mechanism and a feedforward neural network module, and uses residual connections and layer normalization technology to achieve effective connections between modules. S33, the labeled hidden state vector output by the bioBERT model contains deep semantic features and grammatical structure information representing the text, that is, ,in is the vector length of each clinical record, S34. Through multimodal fusion, the semantic features of clinical records and the hierarchical features of ICD codes, as well as the related information between the two, are uniformly represented: ， in, is the bias weight of the co-occurrence matrix, which can be used to control the correlation between clinical text and ICD coding in the entire coding process. S35, based on the idea of ​​gates, combines multiple information sources and optimizes the final target objective function to select which parts of the input are more likely to correctly generate the desired output. Multiplication gates that simultaneously assign importance to various features are used to create a rich multimodal representation. It does not require manual adjustment, but is learned directly from the training data. The final feature vector obtained after gated fusion is: 。 4. The medical code assignment method based on hierarchical association and diversity enhancement according to claim 1 is characterized in that: The specific sub-steps of step S4 are as follows: S41. Since the model ultimately uses a diversified loss function to guide model training, after obtaining the final feature vector of the clinical record text, the CorNet network is used to perform correlation enhancement prediction to generate a related probability matrix: ， in is the label matrix of the training set, is the original prediction probability without enhancement by the CorNet network; S42. In the calculation part of the loss function, first get the most common binary cross entropy loss function ; S43. Input the prediction probability matrix obtained in the CorNet network By selecting the ICD codes corresponding to the Top-K probabilities, the distribution distance of these codes in the ICD tree is calculated. If the predicted ICD codes are concentrated in the same subtree, the loss value increases, forcing the model to disperse the predictions to different subtrees, and obtaining the hierarchical diversity loss ， in That is, in the ICD code hierarchy diagram in the model arrive If the predicted ICD codes are concentrated in the same subtree, the loss value will increase, forcing the model to disperse the predictions to different subtrees. S44. If multiple high-frequency ICD codes have highly similar semantics, their loss values ​​need to be increased to force the model to distinguish similar codes and input the prediction probability matrix With text embedding , calculate the semantic similarity of high-frequency ICD codes and obtain the semantic diversity loss function ; S45. The total loss of the model is determined by these loss functions: 。

Images