A small sample coding method based on nearest neighbor contrastive learning

Abstract

The application discloses a small sample coding method based on nearest neighbor contrast learning, utilizes a convolutional neural network and ICD coding description to capture text semantic features, and utilizes an attention mechanism to capture medical record text content closely related to ICD information, uses an additional structured knowledge unified medical language system and a tree hierarchy of ICD coding to construct positive and negative samples of contrast learning, and then trains a self-CMCL model aiming at different expressions of the same coding, so that the feature representations of synonymous expressions and their abbreviations are closer in a feature space, and expressions of different coding are farther in the feature space. Finally, a k-neighbor module is utilized to integrate existing knowledge in a training set to obtain optimal prediction results.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing, and particularly relates to a small sample coding method based on nearest neighbor contrast learning. BACKGROUND

[0002] International Classification of Diseases (ICD) is a general disease classification method compiled and published by the World Health Organization (WHO). In order to facilitate data storage, retrieval and analysis, ICD converts the diagnosis of health problems into an alphanumeric code. The application of this coding method not only enables disease classification and statistics, but also assists medical institutions in tracking and managing patient diagnosis and treatment. ICD is widely used in multiple fields, including clinical research, medical outcome monitoring, medical payment management and other aspects. In addition, ICD provides a universal medical terminology, using a unified coding standard worldwide, which makes it easy for different hospitals and countries to share and discuss data. This has a profound and extensive impact on the intelligent medical and health service system. According to the white paper of ICD-10 switching ICD-11 reference guide, ICD has been translated into 43 languages and recognized by 194 WHO member states. According to the data in 2017, ICD has been used for death data reporting in 115 countries.

[0003] Firstly, the label distribution of ICD presents a long-tail distribution problem, which brings great challenges to the research. Specifically, the frequently appearing codes (referred to as frequent codes in the present invention) in the clinical electronic medical record only account for a small part of the total number of codes, while the rarely appearing codes (referred to as small sample codes in the present invention) in the clinical electronic medical record account for a large part of the total number of codes. For example, in the medical data set MIMIC-III, there are more than 18,000 ICD-9 codes. After removing the invisible codes, there are 8,922 visible codes. Among them, 5,386 codes only appear 1 to 10 times, and the small sample codes which account for a large part of the code quantity have only a small amount of label data. Secondly, in the electronic medical record, a disease may have multiple expressions or multiple abbreviations, for example, the code description of code 410.01 is "autocoronary coronary atherosclerosis". However, this code can also be described in other text forms, such as "coronary artery disease", "myocardial ischemia", "coronary heart disease", "CAD" and "ASHD". Although a language model can be used to learn the synonymous relationship, previous studies have shown that the language model is only limited to large data sets such as biomedical or clinical knowledge base. For the problem of small sample code data sparsity, the feature representation of multiple different expressions of the same code cannot guarantee that they are close in the feature space, which makes it challenging to accurately classify fine-grained in the biomedical field.

[0004] In real medical applications, common disease types often have sufficient sample quantities, while small sample diseases are relatively few, so they are often ignored or lack sufficient attention in research. However, the ICD codes of small sample diseases also have important practical application value because they are often some rare disease types and have high diagnostic and treatment value for doctors and patients. Therefore, the research on small sample codes is of great significance. SUMMARY

[0005] In view of the above problems in the prior art, the small sample code method based on nearest neighbor contrast learning provided by the present invention solves the problems of high ICD automatic coding error rate caused by long electronic medical record sentences, lack of clinical experience in automatic coding process, and avoids the dependence on manual features.

[0006] In order to achieve the above invention purpose, the technical scheme adopted by the present invention is as follows: a small sample code method based on nearest neighbor contrast learning, comprising the following steps:

[0007] S1, obtaining a character embedding matrix according to the characters in the electronic medical record, and vectorizing the electronic medical record sequence to obtain a feature vector;

[0008] S2, input the feature vector into a convolutional neural network to obtain an upper layer sequence vector, process an ICD code description coded with an encoding to obtain an encoding vector;

[0009] S3, calculate the upper layer sequence vector and the encoding vector by using an attention mechanism to obtain a high-level sequence vector;

[0010] S4, establish a contrast learning loss function according to the high-level sequence vector through a contrast learning technology, train to obtain a CMCL model through the contrast learning loss function, and obtain a contrast learning sequence vector;

[0011] S5, construct a data warehouse, perform k-neighbor prediction according to the contrast learning sequence vector to obtain a nearest neighbor set of the data warehouse, and further obtain a prediction result of the k-neighbor algorithm;

[0012] S6, obtain a prediction result of the CMCL model according to the contrast learning sequence vector, combine the prediction result of the k-neighbor algorithm to perform label judgment to obtain a final prediction result, obtain a prediction label, and complete small sample coding.

[0013] Further, the S1 includes the following steps:

[0014] S11, randomly initialize each character in a character table of an electronic medical record to obtain a character embedding matrix;

[0015] S12, convert each character of an electronic medical record sequence into a character vector, represent the character vector through the vector of the character embedding matrix, and obtain a feature vector.

[0016] Further, in the S11, the expression of the character embedding matrix W is specifically:

[0017]

[0018] In the formula, is the vector of the character embedding matrix, i n is a character ordinal, and i n = 1,...,n, n is a fixed size of an input word table, i d is a dimension ordinal, and i d = 1,...,d, d is a dimension of the character vector;

[0019] In the S12, the feature vector wherein, is the represented character vector, i k is an ordinal of a sequence character, i k = 1,...,k, k is the number of sequence characters.

[0020] Further, the S2 includes the following steps:

[0021] S21, performing a convolution operation on the feature vector through a convolutional neural network to obtain a text feature, and further obtaining an upper layer sequence vector according to the text feature;

[0022] S22, converting words in the encoding description of the ICD code into lowercase, deleting stop words in the encoding description, and obtaining an encoding vector by averaging word embedding vectors of remaining words in the encoding description.

[0023] Further, in the S21, the upper layer sequence vector H = [h1,...,h n ], h1 is an initial text feature, h n is a text feature, and an expression of the text feature h n is specifically as follows:

[0024] h n = f(W c *(x n ,x n+1 ,...,x n+k-1 )+b c )

[0025] In the formula, * is a convolution operator, W c is a weight matrix to be learned, f(·) is a nonlinear transformation, b c represents a bias vector, and (x n ,x n+1 ,...,x n+k-1 ) is a vector input into the convolutional neural network, wherein k is the number of sequence characters, and n is a fixed size of an input vocabulary.

[0026] In the S22, an expression of the encoding vector v i is specifically as follows:

[0027]

[0028] In the formula, N is the number of remaining words in the encoding description, i is the ICD code, x j is a feature vector of the jth word in the encoding description, and s is the number of ICD codes.

[0029] The above further scheme has the beneficial effects that the convolutional neural network is used to accurately extract local core features of a sentence, and a convolution kernel is shared, so that high-dimensional data processing is not stressful, and the encoding description of each code is used to guide the model to appropriate parameters, so as to represent the encoding description of the code.

[0030] Further, the S3 includes the following steps:

[0031] S31, obtaining attention scores of all rows in the upper layer sequence vector according to the upper layer sequence vector and the encoding vector.

[0032] S32, obtaining a specific encoded feature representation according to the attention score of all rows in the upper sequence vector and the upper sequence vector;

[0033] S33, performing an attention operation on the specific encoded feature representation to obtain a high-level sequence vector.

[0034] Further, in the S31, the attention score c of all rows in the upper sequence vector is expressed as: l

[0035]

[0036] wherein, Softmax(·) is a normalized exponential function, tanh(·) is an activation function, is a weight, b a is a bias, H is the upper sequence vector, v i is an encoding vector;

[0037] In the S32, the expression of the specific encoded feature representation a l is specifically:

[0038]

[0039] wherein, is the transposed c l , l is a specific code, and s is the number of ICD codes.

[0040] Further, the S4 comprises the following steps:

[0041] S41, establishing a positive sample of contrast learning;

[0042] The S41 is specifically:

[0043] Mapping medical terms and codes as entities of the unified medical language system, collecting English free text descriptions of the entities through the unified medical language system, establishing entity synonyms and entity abbreviations, and taking the union of the entity synonyms and the entity abbreviations as the positive sample of the contrast learning;

[0044] S42, constructing a negative sample of contrast learning according to a tree-like hierarchical structure of ICD codes;

[0045] S43, establishing an anchor sample according to the high-level sequence vector;

[0046] S44, establishing a contrast learning loss function through the positive sample, the negative sample and the anchor sample, training a model through the contrast learning loss function to obtain a CMCL model, and obtaining a contrast learning sequence vector according to the CMCL model; ​

[0047] wherein the expression of the contrastive learning loss function L(a g ,p g ,n g ) is specifically as follows:

[0048]

[0049] wherein a g is an anchor sample, p g is a positive sample, n g is a negative sample, is a feature representation of the anchor sample, is a feature representation of the positive sample, is a feature representation of the negative sample, is a distance between the anchor sample and the positive sample, is a distance between the anchor sample and the negative sample, and margin is a preset boundary value.

[0050] The above further scheme has the beneficial effect that the contrastive learning loss function established by the present application promotes the model to learn good feature representations by reducing the distance of similar samples in the feature space and simultaneously enlarging the gap between dissimilar samples, so that the model can better distinguish similar and dissimilar samples, thereby achieving better classification and recognition performance.

[0051] Further, the S5 comprises the following sub-steps:

[0052] S51, combining each encoded real class and high-level sequence vector in the medical text of the training set to obtain a data warehouse, wherein the data warehouse D is composed of high-level sequence vectors with a real class of 1 in each medical text in the training set, and the expression is as follows:

[0053] D={(a i1 ,i),(a i2 ,i),...,(a im ,i)} s

[0054] wherein i is an ICD code, s is the number of ICD codes, a i1 is the first high-level sequence vector of the ICD code i, a i2 is the second high-level sequence vector of the ICD code i, and a im is the mth high-level sequence vector of the ICD code i.

[0055] S52, calculate the Euclidean distance between the contrastive learning sequence vector and all feature representations in the data warehouse D, and arrange them in ascending order, and select the top k Euclidean distance corresponding feature representations as the nearest neighbor set, and the nearest neighbor set includes several ICD code corresponding nearest neighbor subsets;

[0056] wherein the expression of the Euclidean distance dis is specifically:

[0057]

[0058] wherein a i is the high-level sequence vector in the data warehouse, b i is the high-level sequence vector of each code in the test set or the validation set;

[0059] S53, obtain the prediction result of the k nearest neighbor algorithm according to the nearest neighbor set, and the method is specifically:

[0060] obtain the ICD code represented by the feature representation with the most nearest neighbor subsets, and judge whether the ICD code is the same as the ICD code corresponding to the nearest neighbor subset;

[0061] if yes, the prediction result of the k nearest neighbor algorithm is The expression of the prediction result of the k nearest neighbor algorithm is specifically:

[0062]

[0063] wherein p is the number of the most feature representations;

[0064] if not, the prediction result of the k nearest neighbor algorithm is 0.

[0065] Further, the S6 comprises the following steps:

[0066] S61, input the contrastive learning sequence vector into the full connection layer to obtain a classifier, and obtain the prediction result of the CMCL model through the classifier, wherein the expression of the prediction result of the CMCL model is specifically:

[0067]

[0068] wherein σ(x) is an activation function, W o represents the weight parameter of the full connection layer, and V is the contrastive learning sequence vector;

[0069] S62, obtain the final prediction result according to the prediction results of the CMCL model and the k nearest neighbor algorithm, wherein the expression of the final prediction result is specifically:

[0070] ​​

[0071] wherein, lambda is a specific gravity parameter, is a prediction result of the k-neighbor algorithm, is a prediction result of the CMCL model;

[0072] S63, a target function is established according to a final prediction result and a correct label, a prediction label is obtained, and small sample coding is completed;

[0073] wherein, the target function The expression of the target function is specifically as follows:

[0074]

[0075] wherein, theta is all training parameters, y i is a real label of ICD coding i, M is the number of real labels, and Y i ∈{0, 1}, L(a g , p g , n g ) is a contrastive learning loss function.

[0076] The beneficial effects of the present application are as follows:

[0077] (1) The small sample coding method based on nearest neighbor contrastive learning provided by the present application can automatically extract features of electronic medical records, does not depend on manual features, and does not completely depend on diseases written by doctors, thereby reducing research time and reducing matching errors.

[0078] (2) The present application trains a CMCL model for different expressions of the same coding through contrastive learning, so that the feature representations of synonymous expressions and their abbreviations are closer in the feature space, and the expressions of different encodings are farther apart in the feature space.

[0079] (3) The present application adopts a convolutional neural network and an attention mechanism operation, can exclude a large amount of irrelevant information in the medical record text, extract local useful information, and further extract text closely related to ICD information by using the attention mechanism.

[0080] (4) The present application fully utilizes the existing feature representation of the training sample through the k-neighbor module, integrates the prediction results of the contrastive learning model, and obtains more accurate prediction results. BRIEF DESCRIPTION OF DRAWINGS

[0081] Figure 1 is a flowchart of the small sample coding method based on nearest neighbor contrastive learning of the present application.

[0082] Figure 2 is a schematic diagram of the ICD automatic coding framework of the present application. DETAILED DESCRIPTION

[0083] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0084] like Figure 1 As shown, in one embodiment of the present invention, a few-shot encoding method based on nearest-neighbor contrastive learning includes the following steps:

[0085] S1. Obtain the character embedding matrix based on the characters in the electronic medical record, and vectorize the electronic medical record sequence to obtain the feature vector;

[0086] S2. Input the feature vector into the convolutional neural network to obtain the upper-layer sequence vector, and process the encoding description of the ICD encoding to obtain the encoding vector;

[0087] S3. Use the attention mechanism to calculate the upper-level sequence vector and encoding vector to obtain the high-level sequence vector;

[0088] S4. Establish a contrastive learning loss function based on the high-level sequence vector using contrastive learning techniques, train the CMCL model using the contrastive learning loss function, and obtain the contrastive learning sequence vector;

[0089] S5. Construct a data warehouse, perform k-nearest neighbor prediction based on the contrastive learning sequence vectors, obtain the nearest neighbor set of the data warehouse, and then obtain the prediction results of the k-nearest neighbor algorithm;

[0090] S6. Based on the contrastive learning sequence vector, obtain the prediction results of the CMCL model, and combine them with the prediction results of the k-nearest neighbor algorithm to determine the label and obtain the final prediction label, thus completing the small sample encoding.

[0091] This invention utilizes convolutional neural networks and ICD encoding to capture semantic features of text, and employs an attention mechanism to capture medical record text content closely related to ICD information. It uses a unified medical language system with additional structured knowledge and a tree-like hierarchical structure of ICD encoding to construct positive and negative samples for contrastive learning, thereby training a self-CMCL model for different expressions of the same encoding. This makes the feature representations of synonyms and their abbreviations closer in the feature space, while the representations of different encodings are farther apart. Finally, the k-nearest neighbor module integrates existing knowledge in the training set to obtain the best prediction results, solving the data sparsity problem of few-sample encoding, which leads to multiple, dissimilar representations of the same encoding in the feature space.

[0092] S1 includes the following steps:

[0093] S11. Randomly initialize each character in the character table of the electronic medical record to obtain the character embedding matrix;

[0094] S12. Convert each character of the electronic medical record sequence into a character vector, and obtain the feature vector by representing the character vector through the character embedding matrix.

[0095] In S11, the expression for the character embedding matrix W is specifically as follows:

[0096]

[0097] In the formula, Let i be a vector of character embedding matrices. n Let i be the ordinal number of the character, and i n =1,...,n, where n is a fixed-size input vocabulary, and i d Let i be the ordinal number of the dimension, and i d =1,...,d, where d is the dimension of the character vector;

[0098] A schematic diagram of the ICD auto-encoding framework of the present invention is shown below. Figure 2 As shown, in this embodiment, all characters in the electronic medical record are extracted after data preprocessing. The extracted characters are deduplicated to form a character table. The required character embedding dimension d is manually set. At the same time, in order to broadly adapt to input characters, the unrecognized character "UNK" is set to replace all characters not in the character table.

[0099] In S12, the feature vector in, For the represented character vector, i k i is the ordinal number of the sequence character. k =1,...,k, where k is the number of characters in the sequence.

[0100] S2 includes the following steps:

[0101] S21. By performing convolution operations on the feature vectors through a convolutional neural network, text features are obtained, and then upper-layer sequence vectors are obtained based on the text features.

[0102] S22. Lowercase the words in the ICD encoding description and remove the stop words in the encoding description. Obtain the encoding vector by averaging the word embedding vectors of the remaining words in the encoding description.

[0103] In S21, the upper-level sequence vector H = [h1,...,h n ], h1 is the initial text feature, h n Text features are obtained by defining text features h. nThe specific expression is:

[0104] h n =f(W c *(x n ,x n+1 ,…,x n+k-1 )+b c )

[0105] In the formula, * represents the convolution operator, W c Let f(·) be the weight matrix to be learned, and b be the nonlinear transformation. c Represents the bias vector, (x n ,x n+1 ,…,x n+k-1 Let be the input vector to the convolutional neural network, where k is the number of characters in the sequence, n is the input vocabulary of a fixed size, and the upper-layer sequence vector H = [h1, h2, h3, ..., hn]. n ],

[0106] Electronic medical records are typically plain text-based unstructured data with extremely long text lengths and a large amount of information unrelated to ICD encoding. Convolutional neural networks can accurately extract local core features of sentences and share convolutional kernels, making high-dimensional data processing stress-free.

[0107] In S22, the encoding vector v i The specific expression is:

[0108]

[0109] In the formula, N is the number of remaining words in the encoding description, i is the ICD encoding, and x is the number of words in the encoding description. j Let be the feature vector of the j-th word in the encoding description, and s be the number of ICD codes.

[0110] Each ICD code has a coding description, such as the coding description of code A15.009 as "proliferative pulmonary tuberculosis". Closely related sibling or parent-child codes have similar coding descriptions. The parameters of a small sample code should be similar to those of frequent codes with similar coding descriptions. The coding description of each code can guide the model towards appropriate parameters. To represent the coding description, this invention preprocesses each coding description. First, all words in the coding description are lowercase and stop words are removed. Then, the coding vector is formed by averaging the word embedding vectors of the remaining words.

[0111] S3 includes the following steps:

[0112] S31. Obtain the attention score for all rows in the upper-level sequence vector based on the upper-level sequence vector and the encoding vector;

[0113] S32. Obtain the feature representation of the specific encoding based on the attention scores of all rows in the upper-level sequence vector and the upper-level sequence vector;

[0114] S33. Perform attention operations on the feature representation of a specific encoding to obtain a high-level sequence vector.

[0115] In S31, the attention score c of all rows in the upper-layer sequence vector l The specific expression is:

[0116]

[0117] In the formula, Softmax(·) is the normalization exponential function, and tanh(·) is the activation function. As the weight, b a Let H be the deviation, H be the upper-level sequence vector, and v be the value of the deviation. i It is an encoded vector;

[0118] In this embodiment, attention scores are used to represent the correlation between the upper-level sequence vector and the encoding vector, thereby assigning multiple codes to each electronic medical record document.

[0119] In step S32, a feature representation a with a specific encoding is obtained. l The specific expression is:

[0120]

[0121] In the formula, c is the transpose l , l represents a specific code, and s represents the number of ICD codes.

[0122] The characteristic representation of a specific encoding l It represents the most relevant text information for a specific code l in H. For each input sample, s feature representations of codes l are generated, each corresponding to a different ICD code. The feature representations of code l are placed into a matrix to form a feature candidate set for a specific code l.

[0123] S4 includes the following sub-steps:

[0124] S41. Establish positive samples for comparative learning;

[0125] Specifically, S41 is:

[0126] A mapping is established between medical terms and codes, which are then used as entities in a unified medical language system. Free English text descriptions of these entities are collected through the unified medical language system. Entity synonyms and abbreviations are then established, and the union of entity synonyms and abbreviations is used as a positive sample for comparative learning.

[0127] In this embodiment, entity synonyms are collected from multiple free-text English descriptions of entities through the "MRCONSO" table of the Unified Medical Language System, and entity abbreviations are collected from multiple free-text English descriptions of entities through the "lrabr" table of the Unified Medical Language System's expert dictionary and lexical tools.

[0128] S42. Construct negative samples for contrastive learning based on the tree-like hierarchical structure encoded by ICD;

[0129] When constructing a negative sample based on ICD code 410.01 (coronary atherosclerosis of autologous coronary arteries), negative samples can be constructed using its parent code 414.0 (coronary atherosclerosis), sibling codes 414.00 (autologous or transplanted coronary atherosclerosis) and 414.04 (coronary atherosclerosis via transarterial bypass grafting), and other unrelated codes, such as 584.9 (acute renal failure).

[0130] S43. Establish anchor point samples based on high-level sequence vectors;

[0131] S44. Establish a contrastive learning loss function using positive samples, negative samples, and anchor samples. Train the model using the contrastive learning loss function to obtain the CMCL model. Obtain the contrastive learning sequence vector based on the CMCL model.

[0132] Among them, the contrastive learning loss function L(a) g ,p g ,n g The specific expression for ) is:

[0133]

[0134] In the formula, a g For anchor point samples, p g For positive samples, n g For negative samples, The feature representation of the anchor point sample, The feature representation of positive samples. For the feature representation of negative samples, The distance between the anchor sample and the positive sample. The distance between the anchor sample and the negative sample is denoted as _margin_, which is a pre-defined boundary value used to control that the distance between the anchor sample and the negative sample is greater than the distance between the anchor sample and the positive sample. The value of _margin_ should be large enough to effectively reduce the distance between pairs of samples of the same class and pairs of samples of different classes during training. If the _margin_ is too small, the distance between pairs of samples of the same class may not be enough to distinguish them, thus reducing the accuracy of the model. If the _margin_ is too large, the distance between pairs of samples of different classes may be too large, making the representations learned by the model too scattered, which will also reduce the accuracy of the model. In this embodiment, the value of _margin_ is between 0.1 and 1.

[0135] In this embodiment, the contrastive learning loss function is used to promote the model to learn good feature representations by reducing the distance between similar samples in the feature space and increasing the gap between dissimilar samples. In this way, the model can better distinguish between similar and dissimilar samples, thereby achieving better classification and recognition performance.

[0136] S5 includes the following steps:

[0137] S51. Combine the true class and high-level sequence vector of each encoded medical text in the training set to obtain the data warehouse. The data warehouse D consists of the high-level sequence vector of each medical text in the training set where the true class is 1. Its expression is:

[0138] D={(a i1 ,i),(a i2 ,i),...,(a im ,i)} s

[0139] In the formula, i is the ICD code, s is the number of ICD codes, and a i1 Let a be the first high-level sequence vector of ICD encoding i. i2 a is the second high-level sequence vector of ICD encoding i. im This is the m-th high-level sequence vector of ICD encoding i;

[0140] In this embodiment, for each training medical text, CMCL is used to obtain the feature representation of each code in the medical text. i ∈{0,1} is the true class of the ICD encoding of i in the training medical text d, and the data warehouse D s From each medical text in the training set, y i The characteristics represent the composition.

[0141] S52. Calculate the Euclidean distance between the contrast learning sequence vector and all feature representations in the data warehouse D, sort them in ascending order, and select the feature representations corresponding to the first k Euclidean distances as the nearest neighbor set. The nearest neighbor set includes several nearest neighbor subsets corresponding to ICD codes.

[0142] The expression for the Euclidean distance *dis* is as follows:

[0143]

[0144] In the formula, a i b is a high-level sequence vector in a data warehouse. i For each encoded high-level sequence vector in the test set or validation set;

[0145] S53. The prediction results of the k-nearest neighbor algorithm are obtained from the nearest neighbor set. The specific method is as follows:

[0146] Obtain the ICD code of the feature representation with the most nearest subset, and determine whether the ICD code is the same as the ICD code corresponding to the nearest subset.

[0147] If so, then the prediction result of the k-nearest neighbor algorithm The specific expression is:

[0148]

[0149] In the formula, p represents the maximum number of feature representations;

[0150] If not, the prediction result of the k-nearest neighbor algorithm is 0.

[0151] In this embodiment, for any given code, the ICD code represented by the feature representation with the largest number of occurrences is selected from the set of codes, and its number is p. If there are multiple feature representations with the largest number of occurrences, the ICD code represented by the feature representation closest to the nearest neighbor is selected as the prediction of the k-nearest neighbor algorithm.

[0152] If the encoding and the sum are the same encoding, then the k-nearest neighbor algorithm's prediction result for that encoding is... If the encoding and q are different encodings, then the prediction result of the k-nearest neighbor algorithm is...

[0153] S6 includes the following sub-steps:

[0154] S61. Input the contrastive learning sequence vector into the fully connected layer to obtain a classifier. Use the classifier to obtain the prediction results of the CMCL model. The prediction results of the CMCL model... The specific expression is:

[0155]

[0156] In the formula, σ(x) is the activation function, and W o This represents the weight parameters of the fully connected layer, and V is the contrastive learning sequence vector.

[0157] S62. Based on the prediction results of the CMCL model and the k-nearest neighbor algorithm, the final prediction result is obtained, wherein the final prediction result... The specific expression is:

[0158]

[0159] In the formula, λ is the specific gravity parameter. The prediction results are from the k-nearest neighbor algorithm. The prediction results are from the CMCL model.

[0160] S63. Based on the final prediction results and the correct labels, establish the objective function, obtain the predicted labels, and complete the small sample coding;

[0161] Wherein, objective function The specific expression is:

[0162]

[0163] In the formula, θ represents all training parameters, and y i Let Y be the actual label with ICD encoding i, M be the number of actual labels, and Y be the actual label with ICD encoding i. i ∈{0,1},L(a f ,p g ,n g ) represents the contrastive learning loss function.

[0164] The beneficial effects of this invention are as follows: The few-sample encoding method based on nearest neighbor contrast learning provided by this invention can automatically extract features from electronic medical records, without relying on manual features or doctors' handwritten disease information, thereby reducing research time and matching errors.

[0165] This invention trains a CMCL model for different representations of the same encoding through comparative learning, making the feature representations of synonyms and their abbreviations closer in the feature space, and the representations of different encodings farther apart in the feature space.

[0166] This invention employs convolutional neural networks and attention mechanisms to extract locally useful information from medical record texts by excluding a large amount of irrelevant information. The attention mechanism can further extract text closely related to ICD information.

[0167] This invention fully utilizes the existing feature representations of training samples through the k-nearest neighbor module and integrates the prediction results of the contrastive learning model to obtain more accurate prediction results.

[0168] In the description of this invention, it should be understood that the terms "center," "thickness," "upper," "lower," "horizontal," "top," "bottom," "inner," "outer," and "radial," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying the relative importance or the number of technical features implicitly specified. Therefore, a feature defined by "first," "second," and "third" may explicitly or implicitly include one or more of that feature.

Claims

1. A few-shot coding method based on nearest-neighbor contrastive learning, characterized in that, Includes the following steps: S1. Obtain the character embedding matrix based on the characters in the electronic medical record, and vectorize the electronic medical record sequence to obtain the feature vector; S2. Input the feature vector into the convolutional neural network to obtain the upper-layer sequence vector, and process the encoding description of the ICD encoding to obtain the encoding vector; S3. Use the attention mechanism to calculate the upper-level sequence vector and encoding vector to obtain the high-level sequence vector; S4. Establish a contrastive learning loss function based on the high-level sequence vector using contrastive learning techniques, train the CMCL model using the contrastive learning loss function, and obtain the contrastive learning sequence vector; S5. Construct a data warehouse and perform comparative learning of sequence vectors. k Nearest neighbor prediction yields the nearest neighbor set of the data warehouse, and thus... k The prediction results of the nearest neighbor algorithm; S6. Obtain the prediction results of the CMCL model based on the contrastive learning sequence vectors, and combine them with... k The prediction results of the nearest neighbor algorithm are used to determine the label and obtain the final prediction result, thus completing the small sample encoding.

2. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 1, characterized in that, S1 includes the following steps: S11. Randomly initialize each character in the character table of the electronic medical record to obtain the character embedding matrix; S12. Convert each character of the electronic medical record sequence into a character vector, and obtain the feature vector by representing the character vector through the character embedding matrix.

3. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 2, characterized in that, In S11, the character embedding matrix W The specific expression is: In the formula, A vector of character embedding matrices. It is the ordinal number of the character, and , n For a fixed-size input vocabulary, Let be the ordinal number of the dimension, and , d The dimension of the character vector; In S12, the feature vector ,in, The represented character vector, The ordinal number of the sequence character. , k The number of characters in the sequence.

4. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 1, characterized in that, S2 includes the following steps: S21. By performing convolution operations on the feature vectors through a convolutional neural network, text features are obtained, and then upper-layer sequence vectors are obtained based on the text features. S22. Lowercase the words in the ICD encoding description and remove the stop words in the encoding description. Obtain the encoding vector by averaging the word embedding vectors of the remaining words in the encoding description.

5. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 4, characterized in that, In S21, the upper-level sequence vector , As initial text features, Text features are obtained by using text features. The specific expression is: In the formula, For convolution operators, The weight matrix to be learned. It is a nonlinear transformation. This represents the bias vector. Let be the vector input to the convolutional neural network, where k The number of characters in the sequence. n A fixed-size input vocabulary; In S22, the encoding vector v i The specific expression is: In the formula, N To encode the number of remaining words, i ICD encoding, x j For the first in the encoding description j Feature vectors of each word s This represents the number of ICD codes.

6. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 1, characterized in that, S3 includes the following steps: S31. Obtain the attention score for all rows in the upper-level sequence vector based on the upper-level sequence vector and the encoding vector; S32. Obtain the feature representation of the specific encoding based on the attention scores of all rows in the upper-level sequence vector and the upper-level sequence vector; S33. Perform attention operations on the feature representation of a specific encoding to obtain a high-level sequence vector.

7. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 6, characterized in that, In S31, the attention score of all rows in the upper-layer sequence vector c l The specific expression is: In the formula, For normalized exponential functions, For activation function, As weight, For deviation, H The upper-level sequence vector, v i It is an encoded vector; In step S32, a feature representation with a specific encoding is obtained. a l The specific expression is: In the formula, For transpose c l , l For a specific encoding, s This represents the number of ICD codes.

8. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 1, characterized in that, S4 includes the following sub-steps: S41. Establish positive samples for comparative learning; Specifically, S41 is: A mapping is established between medical terms and codes, which are then used as entities in a unified medical language system. Free English text descriptions of these entities are collected through the unified medical language system. Entity synonyms and abbreviations are then established, and the union of entity synonyms and abbreviations is used as a positive sample for comparative learning. S42. Construct negative samples for contrastive learning based on the tree-like hierarchical structure encoded by ICD; S43. Establish anchor point samples based on high-level sequence vectors; S44. Establish a contrastive learning loss function using positive samples, negative samples, and anchor samples. Train the model using the contrastive learning loss function to obtain the CMCL model. Obtain the contrastive learning sequence vector based on the CMCL model. Among them, the contrastive learning loss function The specific expression is: In the formula, a For anchor point samples, p As a positive sample, g For negative samples, ICD encoding The high-level sequence vector of the anchor sample, ICD encoding The high-level sequence vector of positive samples, ICD encoding The high-level sequence vector of the negative samples, For encoding The distance between the anchor point sample and the positive sample. ICD encoding The distance between anchor samples and negative samples. These are pre-defined boundary values.

9. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 1, characterized in that, S5 includes the following steps: S51. Combine the true class and high-level sequence vector of each encoding in the medical texts of the training set to obtain a data warehouse. It consists of high-level sequence vectors where the true class is 1 in each medical text in the training set, and its expression is: In the formula, ICD encoding, Number of ICD codes ICD encoding i The first high-level sequence vector, ICD encoding i The second high-level sequence vector, ICD encoding i The m A high-level sequence vector; S52, Calculate contrastive learning sequence vectors and data warehouse Calculate the Euclidean distances of all feature representations in the dataset, sort them in ascending order, and select the top [values]. k The feature representations corresponding to each Euclidean distance are used as the nearest neighbor set, which includes several nearest neighbor subsets corresponding to ICD codes. Among them, Euclidean distance dis The specific expression is: In the formula, a i For high-level sequence vectors in a data warehouse, b i For each encoded high-level sequence vector in the test set or validation set; S53, obtained from the nearest neighbor set k The prediction results of the nearest neighbor algorithm are specifically described as follows: Obtain the ICD code of the feature representation with the most nearest subset, and determine whether the ICD code is the same as the ICD code corresponding to the nearest subset. If so, then k Prediction results of the nearest neighbor algorithm The specific expression is: In the formula, p The maximum number of features represented; If not, then k The nearest neighbor algorithm's prediction result is 0.

10. The few-shot coding method based on nearest-neighbor contrastive learning according to claim 9, characterized in that, S6 includes the following sub-steps: S61. Input the contrastive learning sequence vector into the fully connected layer to obtain a classifier. Use the classifier to obtain the prediction results of the CMCL model. The prediction results of the CMCL model... The specific expression is: In the formula, For activation function, This represents the weight parameters of the fully connected layer. V To learn sequence vectors by comparison; S62, Based on the CMCL model and k The nearest neighbor algorithm yields the final prediction result, where the final prediction result... The specific expression is: In the formula, This is a specific gravity parameter. for k The prediction results of the nearest neighbor algorithm, The prediction results of the CMCL model; S63. Based on the final prediction results and the correct labels, establish the objective function, obtain the predicted labels, and complete the small sample coding; Wherein, objective function The specific expression is: In the formula, For all training parameters, ICD encoding The true label, M The number of real labels, and , To compare the learning loss function.

Images