A pathological text classification method

Abstract

The application provides a pathological text classification method, comprising the following steps: S1: according to a preset quantity threshold, labels of a first pathological text data set for training are divided into dense labels and sparse labels, corresponding to constitute a large class label set and a small class label set; S2: a pre-constructed dense label prediction network and a sparse label prediction network are trained by using the first pathological text data set, to obtain a trained dense label prediction network and a trained sparse label prediction network; S3: a second pathological text data set to be classified is respectively input into the trained dense label prediction network and the trained sparse label prediction network, to correspondingly obtain a first prediction result and a second prediction result; and S4: the prediction result of the dense label and the prediction result of the sparse label are fused, to obtain a final prediction result as a classification result. The application provides a pathological text classification method, and solves the problem that the classification effect of pathological text with a small amount of sample data is poor at present.

Description

Technical Field

[0001] This invention relates to the technical field of text classification, and more specifically, to a method for classifying pathological texts. Background Technology

[0002] Pathological text data is obtained through pathological diagnosis and involves multiple stages. Due to the differences among doctors in regional hospitals, coupled with the inherent special nature of medicine, the overall pathological text data is not standardized. Pathological diagnosis is made by the pathology center after sampling, embedding, and preparing the submitted tissue, and then observing the stained slides. Submitted data is entered by clinical hospitals, but the text styles and other aspects of data entry vary from hospital to hospital, making it unstructured data. Pathological text data includes patient information, hospital department, examination items, submitted sites, gross findings, and diagnostic information. Due to its common characteristics—different data sources, relative non-standardization, varying lengths, a combination of Chinese and English, special meanings, medical terminology, and weak semantic information (though containing some semantic information)—multi-label classification of pathological texts is difficult.

[0003] Different pathologies have varying probabilities of occurrence; the higher the probability of a pathology's occurrence, the larger the sample size, and vice versa. Therefore, the pathology text data obtained by hospital pathology centers is often imbalanced. Currently, the most common approach is to classify pathology texts using neural network classification models. High-incidence diseases, with their large sample sizes, allow their semantic features to be fully learned, often resulting in good classification performance. However, pathology text datasets inevitably contain some pathology texts with weak semantic information and small sample sizes. The features of these texts are difficult to learn fully, easily leading to misclassification or missed classification, resulting in poor classification performance. Summary of the Invention

[0004] This invention provides a pathological text classification method to overcome the current technical shortcomings of poor classification performance for pathological texts with limited sample data.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A pathological text classification method includes the following steps:

[0007] S1: Based on a preset threshold, the labels of the first pathological text dataset used for training are divided into dense labels and sparse labels, which correspond to the set of major category labels and the set of minor category labels.

[0008] S2: Train a pre-constructed dense label prediction network and a sparse label prediction network using the first pathological text dataset to obtain the trained dense label prediction network and sparse label prediction network.

[0009] S3: Input the second pathological text dataset to be classified into the trained dense label prediction network and sparse label prediction network respectively to obtain the first prediction result and the second prediction result.

[0010] The first prediction result includes the prediction results of dense labels and the prediction results of subclass label sets;

[0011] The second prediction result includes the prediction results of sparse labels and the prediction results of the large category label set;

[0012] S4: The prediction results of dense labels and sparse labels are fused together to obtain the final prediction result as the classification result.

[0013] In the above scheme, the labels are divided into a set of major labels and a set of minor labels. Dense label prediction network and sparse label prediction network are used to classify and predict pathological texts respectively. This solves the problem of insufficient sample data by transferring it to the sparse label prediction network. Finally, the accurate final prediction result is obtained based on the prediction results of the two prediction networks, thus improving the classification effect of pathological texts.

[0014] Preferably, in step S1, the number of labels in the first pathological text dataset and the number of samples corresponding to each label are counted. Labels with fewer than the number of samples are classified as sparse labels to form a set of small class labels, and labels with more than the number of samples are classified as dense labels to form a set of large class labels.

[0015] Preferably, the dense label prediction network includes a first pre-trained language model, an attention module, a convolutional network module, and a first multilayer perceptron;

[0016] First, the pathological text is input into the first pre-trained language model, which outputs pathological text word vectors. Then, the pathological text word vectors are input into the attention module to obtain the word vector features represented by the attention. The word vector features are then input into the convolutional network module, which uses four convolutional layers for feature extraction. The features extracted by each convolutional layer are used as the input of the next layer. The features extracted by the four convolutional layers are fused to obtain fused word vectors. Finally, the fused word vectors are converted into prediction scores for the corresponding labels through a multilayer perceptron.

[0017] Preferably, the convolutional network module is a multi-character feature convolutional fusion module, which is used to sequentially extract single-character word vectors, two-character word vectors, three-character word vectors and four-character word vectors of size L*D based on the word vector features of dimension D, and finally fuse the single-character word vectors, two-character word vectors, three-character word vectors and four-character word vectors of size L*D to obtain a multi-character feature fused word vector of size L*D; L is the maximum sentence length.

[0018] Preferably, the sparse label prediction network includes a second pre-trained language model and a second multilayer perceptron;

[0019] Before inputting the pathological text into the second pre-trained language model, the process also includes: setting one or more corresponding prompt learning templates based on the semantic relationship between the pathological text to be input and the labels, and adding the pathological text to be input to the prompt learning templates.

[0020] The prompt learning template is based on a specific symptom T: A; where T is the input pathological text and A is the text answer to be predicted.

[0021] After adding the pathological text to be input to the prompt learning template, input it into the second pre-trained language model. Use the second pre-trained language model to encode it to obtain the word vector of the answer to be predicted. The word vectors of different answers are input into the second multilayer perceptron to obtain the prediction score of the corresponding label.

[0022] Preferably, the number of learning prompts is three, the difference being that the reserved text answer to be predicted has different character lengths, namely 2, 3, and 4 characters.

[0023] Preferably, it also includes converting the predicted score of the label into the predicted probability of the label:

[0024]

[0025] Among them, y i This represents the predicted score for the i-th label.

[0026] Preferably, the dense label prediction network uses cross-entropy as the loss function L1:

[0027]

[0028] Where m+1 represents the number of label categories that need to be predicted in the dense label prediction network, including m dense labels and 1 set of subclass labels.

[0029] Preferably, the sparse label prediction network uses cross-entropy as the loss function L2:

[0030]

[0031] Wherein, n+1 represents the number of label categories that need to be predicted in the sparse label prediction network, including n sparse labels and 1 set of major label categories.

[0032] Preferably, the total loss function L3 is the sum of the losses of the dense label prediction network and the sparse label prediction network:

[0033] L3 = L1 + L2

[0034] Where L1 is the loss function of the dense label prediction network and L2 is the loss function of the sparse label prediction network.

[0035] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0036] This invention provides a pathological text classification method. By dividing the labels into a set of major labels and a set of minor labels, and using a dense label prediction network and a sparse label prediction network to classify and predict the pathological texts respectively, the problem of insufficient sample data is transferred to the sparse label prediction network for resolution. Finally, an accurate final prediction result is obtained based on the prediction results of the two prediction networks, thereby improving the classification effect of pathological texts. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the implementation steps of the technical solution of the present invention;

[0038] Figure 2 This is a flowchart illustrating the training process of the dense label prediction network and the sparse label prediction network in this invention.

[0039] Figure 3 This is a flowchart of the prediction process for the dense label prediction network and the sparse label prediction network in this invention.

[0040] Figure 4 This is a schematic diagram of the network structure of the multi-character feature convolutional fusion module in this invention. Detailed Implementation

[0041] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0042] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0043] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0044] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] Example 1

[0046] like Figure 1-3 As shown, a pathological text classification method includes the following steps:

[0047] S1: Based on a preset threshold, the labels of the first pathological text dataset used for training are divided into dense labels and sparse labels, which correspond to the set of major category labels and the set of minor category labels.

[0048] S2: Train a pre-constructed dense label prediction network and a sparse label prediction network using the first pathological text dataset to obtain the trained dense label prediction network and sparse label prediction network.

[0049] S3: Input the second pathological text dataset to be classified into the trained dense label prediction network and sparse label prediction network respectively to obtain the first prediction result and the second prediction result.

[0050] The first prediction result includes the prediction results of dense labels and the prediction results of subclass label sets;

[0051] The second prediction result includes the prediction results of sparse labels and the prediction results of the large category label set;

[0052] S4: The prediction results of dense labels and sparse labels are fused together to obtain the final prediction result as the classification result.

[0053] In the specific implementation process, the labels are divided into a set of major labels and a set of minor labels. Dense label prediction network and sparse label prediction network are used to classify and predict pathological texts respectively. This solves the problem of insufficient sample data by transferring it to the sparse label prediction network. Finally, the accurate final prediction result is obtained based on the prediction results of the two prediction networks, which improves the classification effect of pathological texts.

[0054] Example 2

[0055] A pathological text classification method includes the following steps:

[0056] S1: Based on a preset threshold, the labels of the first pathological text dataset used for training are divided into dense labels and sparse labels, which correspond to the set of major category labels and the set of minor category labels.

[0057] S2: Train a pre-constructed dense label prediction network and a sparse label prediction network using the first pathological text dataset to obtain the trained dense label prediction network and sparse label prediction network.

[0058] S3: Input the second pathological text dataset to be classified into the trained dense label prediction network and sparse label prediction network respectively to obtain the first prediction result and the second prediction result.

[0059] The first prediction result includes the prediction results of dense labels and the prediction results of subclass label sets;

[0060] The second prediction result includes the prediction results of sparse labels and the prediction results of the large category label set;

[0061] S4: The prediction results of dense labels and sparse labels are fused together to obtain the final prediction result as the classification result.

[0062] More specifically, in step S1, the number of labels in the first pathological text dataset and the number of samples corresponding to each label are counted. Labels with fewer than a certain number of samples are classified as sparse labels, forming a set of subclass labels S (where s i For specific sparse labels, labels with a sample size no less than a threshold are divided into dense labels, forming a large category label set L (where l is a specific type of label). i (For specific dense tags).

[0063] In the specific implementation process, sparse labels and dense labels are divided according to the number of samples. Sparse labels are merged into a set of subclass labels, and dense labels are merged into a set of major class labels. For example, in one embodiment, samples with fewer than 50 samples are designated as sparse labels, resulting in a set of subclass labels S = {s1, s2, ..., s...} n ; n = 30}, the set of category labels L = {l1, l2, ... l m ;m=24}.

[0064] More specifically, the dense label prediction network includes a first pre-trained language model, an attention module, a convolutional network module, and a first multilayer perceptron;

[0065] First, the pathological text is input into the first pre-trained language model, and the output is the learned pathological text word vectors. Then, the pathological text word vectors are input into the attention module to learn the attention degree between different words in the sentence, and obtain the attention representation word vector features. The word vector features are input into the convolutional network module, and four convolutional layers are used for feature extraction. The features extracted by each convolutional layer are used as the input of the next layer. The features extracted by the four convolutional layers are fused to obtain fused word vectors. Finally, the fused word vectors are converted into prediction scores for corresponding labels through a multilayer perceptron.

[0066] More specifically, such as Figure 4As shown, the convolutional network module is a multi-character feature convolutional fusion module, used to sequentially extract single-character word vectors, two-character word vectors, three-character word vectors, and four-character word vectors of size L*D based on the word vector features of dimension D. Finally, the single-character word vectors, two-character word vectors, three-character word vectors, and four-character word vectors of size L*D are fused to obtain a multi-character feature fused word vector of size L*D; L is the maximum sentence length; each convolutional layer sets D 2*D convolutional kernel templates to keep the input and output dimensions unchanged; since the size of each convolutional kernel is 2*D, each convolutional layer can be regarded as a feature fusion between two characters; each layer outputs a set of vectors with the same dimension but different content to represent text features with different numbers of characters; finally, the initial input and the word vectors output by each layer are added to obtain the final output.

[0067] In the specific implementation process, text features are extracted through the multi-character feature convolution fusion module. The word vectors after feature fusion can better capture key information such as the diseased area.

[0068] More specifically, this also includes converting the predicted scores of labels into predicted probabilities of labels:

[0069]

[0070] Among them, y i This represents the predicted score for the i-th label.

[0071] In the specific implementation process, the first multilayer perceptron uses a sigmoid layer to convert the label prediction score into the label prediction probability. By converting the label prediction score output by the dense label prediction network into the label prediction probability, a first prediction result is obtained. The first prediction result includes the prediction results of dense labels and the prediction results of the minor category label set; the second prediction result includes the prediction results of sparse labels and the prediction results of the major category label set; the first prediction result and the second prediction result are then fused after removing the prediction results of the minor category label set and the prediction results of the major category label set, respectively, to obtain the final prediction result.

[0072] More specifically, the dense label prediction network uses cross-entropy as the loss function L1:

[0073]

[0074] Where m+1 represents the number of label categories to be predicted in the dense label prediction network, including m dense labels {l1, l2, ..., l...} m} and a set of 1 subclass labels.

[0075] Example 3

[0076] A pathological text classification method includes the following steps:

[0077] S1: Based on a preset threshold, the labels of the first pathological text dataset used for training are divided into dense labels and sparse labels, which correspond to the set of major category labels and the set of minor category labels.

[0078] S2: Train a pre-constructed dense label prediction network and a sparse label prediction network using the first pathological text dataset to obtain the trained dense label prediction network and sparse label prediction network.

[0079] S3: Input the second pathological text dataset to be classified into the trained dense label prediction network and sparse label prediction network respectively to obtain the first prediction result and the second prediction result.

[0080] The first prediction result includes the prediction results of dense labels and the prediction results of subclass label sets;

[0081] The second prediction result includes the prediction results of sparse labels and the prediction results of the large category label set;

[0082] S4: The prediction results of dense labels and sparse labels are fused together to obtain the final prediction result as the classification result.

[0083] More specifically, the sparse label prediction network includes a second pre-trained language model and a second multilayer perceptron;

[0084] Before inputting the pathological text into the second pre-trained language model, the process also includes: setting one or more corresponding prompt learning templates based on the semantic relationship between the pathological text to be input and the labels, and adding the pathological text to be input to the prompt learning templates.

[0085] The prompt learning template is based on a specific symptom T: A; where T is the input pathological text and A is the text answer to be predicted.

[0086] After adding the pathological text to be input to the prompt learning template, input it into the second pre-trained language model. Use the second pre-trained language model to encode it to obtain the word vector of the answer to be predicted. The word vectors of different answers are input into the second multilayer perceptron to obtain the prediction score of the corresponding label.

[0087] In practice, traditional text classification networks still struggle to learn effectively from small datasets due to the limited number of labeled data samples required for sparse prediction tasks. Leveraging the prior knowledge of pre-trained language models can mitigate this data shortage to some extent. Hint templates can be designed based on the dataset, and these templates, encoded by the pre-trained language model, can yield word vectors for the predicted text answer.

[0088] More specifically, there are three learning templates, differing in the length of the text answer to be predicted: 2, 3, and 4 characters respectively. Since the length of the text answer to be predicted is unknown, while the actual affected area is typically 2 to 4 characters long, three learning templates will be used, with 2 to 4 spaces reserved for the text answer position respectively.

[0089] In the implementation process, a cue learning method was used for labels with limited sample data, fully utilizing the prior knowledge of the pre-trained language model to compensate for classification errors caused by insufficient data. Simultaneously, three cue templates were designed for the key information of the diseased area, with different text answer lengths set within each template, to increase the diversity of the predicted text answers. The label prediction scores obtained from different answers were accumulated before making a final prediction, improving the accuracy of sparse prediction.

[0090] More specifically, this also includes converting the predicted scores of labels into predicted probabilities of labels:

[0091]

[0092] Among them, y i This represents the predicted score for the i-th label.

[0093] In the specific implementation, the second multilayer perceptron uses a sigmoid layer to convert the label prediction score into the label prediction probability. By converting the label prediction score output by the sparse label prediction network into the label prediction probability, a second prediction result is obtained. The first prediction result includes the prediction results of dense labels and the prediction results of the minor category label set; the second prediction result includes the prediction results of sparse labels and the prediction results of the major category label set. The first and second prediction results are then fused after removing the prediction results of the minor category label set and the prediction results of the major category label set, respectively, to obtain the final prediction result.

[0094] More specifically, the sparse label prediction network uses cross-entropy as the loss function L2:

[0095]

[0096] Where n+1 represents the number of label categories to be predicted in the sparse label prediction network, including n sparse labels {s1, s2, ..., sn}. n} and a set of 1 major category tags.

[0097] More specifically, the first and second pre-trained language models share parameters, and the total loss function L3 is the sum of the losses of the dense label prediction network and the sparse label prediction network.

[0098] L3 = L1 + L2

[0099] Here, L1 is the loss function of the dense label prediction network, and L2 is the loss function of the sparse label prediction network. By taking the gradient derivative of the total loss function, the two pre-trained language models ALBERT are jointly learned through backpropagation, while the parameters of other network modules are updated.

[0100] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method of pathological text classification, characterized by, Includes the following steps: S1: Based on a preset threshold, the labels of the first pathological text dataset used for training are divided into dense labels and sparse labels, which correspond to the set of major category labels and the set of minor category labels. S2: Train a pre-constructed dense label prediction network and a sparse label prediction network using the first pathological text dataset to obtain the trained dense label prediction network and sparse label prediction network. S3: Input the second pathological text dataset to be classified into the trained dense label prediction network and sparse label prediction network respectively to obtain the first prediction result and the second prediction result. The first prediction result includes the prediction results of dense labels and the prediction results of subclass label sets; The second prediction result includes the prediction results of sparse labels and the prediction results of the large category label set; S4: The prediction results of dense labels and sparse labels are fused together to obtain the final prediction result as the classification result; In step S1, the number of labels in the first pathological text dataset and the number of samples corresponding to each label are counted. Labels with fewer than the number threshold are classified as sparse labels and form a set of small class labels. Labels with more than the number threshold are classified as dense labels and form a set of large class labels. The dense label prediction network includes a first pre-trained language model, an attention module, a convolutional network module, and a first multilayer perceptron; First, the pathological text is input into the first pre-trained language model, which outputs pathological text word vectors. Then, the pathological text word vectors are input into the attention module to obtain the word vector features represented by the attention. The word vector features are input into the convolutional network module, which uses four convolutional layers for feature extraction. The features extracted by each convolutional layer are used as the input of the next layer. The features extracted by the four convolutional layers are fused to obtain fused word vectors. Finally, the fused word vectors are converted into prediction scores for the corresponding labels through a multilayer perceptron. The sparse label prediction network includes a second pre-trained language model and a second multilayer perceptron. Before inputting the pathological text into the second pre-trained language model, the process also includes: setting one or more corresponding prompt learning templates based on the semantic relationship between the pathological text to be input and the labels, and adding the pathological text to be input to the prompt learning templates. The prompt learning template is based on a specific symptom T: A; where T is the input pathological text and A is the text answer to be predicted. After adding the pathological text to be input to the prompt learning template, input it into the second pre-trained language model. Use the second pre-trained language model to encode it to obtain the word vector of the answer to be predicted. The word vectors of different answers are input into the second multilayer perceptron to obtain the prediction score of the corresponding label.

2. The method of pathological text classification according to claim 1, characterized in that, The convolutional network module is a multi-character feature convolutional fusion module, which is used to sequentially extract single-character word vectors, two-character word vectors, three-character word vectors and four-character word vectors of size L*D based on the word vector features of dimension D. Finally, the single-character word vectors, two-character word vectors, three-character word vectors and four-character word vectors of size L*D are fused to obtain a multi-character feature fused word vector of size L*D; L is the maximum sentence length.

3. The method of claim 1, wherein the method further comprises: There are three learning templates, the difference being the length of the reserved text answer to be predicted, which is 2, 3, and 4 characters respectively.

4. The method of claim 1, wherein, It also includes converting the predicted scores of the labels into predicted probabilities for the labels: wherein, denotes the predicted score of the i th label.

5. The method of pathological text classification according to claim 4, characterized in that, The dense label prediction network adopts cross entropy as a loss function : wherein, m +1 is the number of label classes to be predicted in the dense label prediction network, including m one dense label and one set of small class labels.

6. The method of pathological text classification according to claim 5, wherein, The sparse label prediction network adopts cross entropy as a loss function : in, n +1 represents the number of label categories that need to be predicted in the sparse label prediction network, including... n A set of sparse labels and a set of 1 major category labels.

7. The method of pathological text classification according to claim 6, characterized in that, Total loss function Losses for dense label prediction network and sparse label prediction network are added: wherein, is a loss function for the dense label prediction network, is a loss function for the sparse label prediction network.

Images