Text classification method and device, electronic equipment and computer readable storage medium

Abstract

The present application relates to the artificial intelligence technology in the field of medical technology, disclose a kind of text classification method based on comparison learning and label information, comprising: using pre-constructed text classification model each training text in training text set is converted into text vector, and the text label corresponding to each training text is converted into label vector, the first loss value between the text vector of training text and the label vector of corresponding text label is calculated, the second loss value between label vector is calculated, the first loss value and the second loss value are fused, and the comprehensive loss value is obtained, the parameter of pre-constructed text classification model is adjusted using comprehensive loss value, until the training condition of comprehensive loss value meets preset, using the text classification model of well-trained text is classified to text to be classified.The present application also proposes a kind of text classification device based on comparison learning and label information, electronic equipment and computer readable storage medium.The present application can improve the accuracy of medical text classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical technology, and in particular to a text classification method and device based on comparison learning and label information, an electronic device and a computer readable storage medium. BACKGROUND

[0002] With the continuous development of deep learning technology, deep learning technology has been widely developed in the field of intelligent medical treatment, especially in the medical classification scene based on medical text information. For example, based on the patient's symptom description, disease history information, etc. to classify diseases, based on medical literature, medical records information to classify patients to assist doctors in diagnosis and treatment decision-making.

[0003] In the above classification application based on medical text, the deep learning-based method can automatically learn the feature representation of the text, has high accuracy and generalization ability, and in the application scene of text classification, classification label information is often introduced, such as lesion classification label information of kidney, lung, etc. Department classification label information such as respiratory department, cardiovascular department, etc. Label information is often used as a classification index, often ignoring the text information of the classification label itself, and the difference between multiple classification labels and the difference between the classification label and the classification text. Therefore, the accuracy of traditional deep learning-based text classification based on labels needs to be further improved. SUMMARY

[0004] The present application provides a text classification method and device based on comparison learning and label information, an electronic device and a computer readable storage medium, which mainly aims to improve the accuracy of medical text classification.

[0005] To achieve the above purpose, the present application provides a text classification method based on comparison learning and label information, comprising:

[0006] Obtain a training text set with pre-labeled text labels, and convert each training text in the training text set into a text vector using a pre-constructed text classification model, and convert the text label corresponding to each training text into a label vector;

[0007] Calculate the first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text, and calculate the second loss value between each label vector, fuse the first loss value and the second loss value, and obtain the comprehensive loss value;

[0008] adjusting parameters of the pre-constructed text classification model by using the comprehensive loss value, and iteratively training the text classification model after the parameter adjustment until the comprehensive loss value meets a preset training condition, to obtain a trained text classification model and a latest label vector of each text label;

[0009] obtaining a text to be classified, and classifying the text to be classified according to the latest label vector of each text label by using the trained text classification model.

[0010] Optionally, the converting each training text in the training text set into a text vector by using the pre-constructed text classification model in sequence comprises:

[0011] performing word segmentation processing on each training text in the training text set to obtain a word segmentation set corresponding to each training text;

[0012] selecting one word segmentation from the word segmentation set as a target word segmentation, and counting a co-occurrence number of the target word segmentation and adjacent word segmentations of the target word segmentation within a preset neighborhood range of the target word segmentation;

[0013] constructing a co-occurrence matrix by using the co-occurrence number corresponding to each word segmentation;

[0014] converting the word segmentation corresponding to each training text into a word vector by using the pre-constructed text classification model, and splicing the word vectors into a vector matrix;

[0015] performing a product operation on the co-occurrence matrix and the vector matrix to obtain a text vector corresponding to each training text.

[0016] Optionally, the sequentially calculating a first loss value between the text vector of each training text and a label vector of a text label corresponding to the training text comprises:

[0017] calculating a first loss value between the text vector of each training text and a label vector of a text label corresponding to the training text by using a first loss function as follows:

[0018]

[0019] wherein, Loss1 represents the first loss value, h i represents a text vector of an i-th text in the training text set, l i represents a label vector of a text label corresponding to the i-th text, sim(h i , l i ) represents a cosine similarity between h i and l i , and l ja label vector representing a jth text label, N represents a number of text labels in the preset text label set, sim (l i ,l i ) represents a cosine similarity between l i and l j , and τ is a temperature coefficient.

[0020] Optionally, the second loss value between each of the label vectors is calculated, including:

[0021] The second loss value between each of the label vectors is calculated by using a second loss function as follows:

[0022]

[0023] wherein, Loss2 represents the second loss value, i and j respectively represent an ith text label and a jth text label in the preset text label set, and i is not equal to j, l j represents a jth text label vector, l i represents an ith text label vector, sim (l i , l j ) represents a cosine similarity between l i and l j .

[0024] Optionally, the first loss value and the second loss value are fused to obtain a comprehensive loss value, including:

[0025] A preset first weight corresponding to the first loss value and a second weight corresponding to the second loss value are obtained;

[0026] The first loss value and the second loss value are weighted and averaged by using the first weight and the second weight to obtain the comprehensive loss value.

[0027] Optionally, the text to be classified is classified according to the nearest label vector of each of the text labels by using the trained text classification model, including:

[0028] The text to be classified is converted into a text vector to be classified by using the trained text classification model;

[0029] A probability value between the text vector to be classified and the nearest label vector of each of the text labels is calculated;

[0030] A text label corresponding to a probability value satisfying a preset classification condition is selected as a classification label of the text to be classified.

[0031] Optionally, the calculating the probability value between the text vector to be classified and the nearest label vector of each text label comprises:

[0032] sequentially calculating the cosine similarity between the text vector to be classified and the nearest label vector of each text label;

[0033] normalizing all the cosine similarities, and converting the normalized cosine similarities into the probability values.

[0034] To solve the above problems, the application further provides a text classification device based on alignment learning and label information, which comprises:

[0035] a text and label vector representation module, configured to acquire a training text set with pre-labeled text labels, and sequentially convert each text in the training text set into a text vector and each text label in the preset text label set into a label vector by using a pre-built text classification model;

[0036] a text and label loss calculation module, configured to sequentially calculate a first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text, and a second loss value between each label vector, and fuse the first loss value and the second loss value to obtain a comprehensive loss value;

[0037] a classification model training module, configured to adjust the parameters of the pre-built text classification model by using the comprehensive loss value, and iteratively train the text classification model after the parameter adjustment until the comprehensive loss value meets a preset training condition, thereby obtaining a trained text classification model and the nearest label vector of each text label;

[0038] a text to be classified classification module, configured to acquire a text to be classified, and classify the text to be classified according to the nearest label vector of each text label by using the trained text classification model.

[0039] To solve the above problems, the application further provides an electronic device, which comprises:

[0040] a memory, configured to store at least one computer program; and

[0041] a processor, configured to execute the program stored in the memory to implement the above-mentioned text classification method based on alignment learning and label information.

[0042] To solve the above problems, the application further provides a computer readable storage medium, wherein at least one computer program is stored in the computer readable storage medium, and the at least one computer program is executed by a processor in an electronic device to implement the text classification method based on comparison learning and label information.

[0043] The application drives the alignment of the training text and the corresponding text label in the semantic space by calculating the first loss value between the text vector of each training text in the training text set composed of medical text information and the label vector of the medical text label corresponding to the training text, and at the same time, calculates the second loss value between each label vector to avoid the semantic similarity of the text label, thereby avoiding the problem of blurred boundary of text classification. Finally, the parameters of the pre-constructed text classification model are adjusted by using the comprehensive loss value corresponding to the first loss value and the second loss value, and the pre-constructed text classification model is trained, so that the model can sufficiently learn the meaning of the training text and the meaning of the text label, and then the classification of the to-be-classified text is performed by using the trained text classification model, which can improve the accuracy of medical text classification. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 A flowchart of a text classification method based on comparison learning and label information provided by an embodiment of the application is shown in the figure.

[0045] Figure 2 A detailed implementation flowchart of one step of the text classification method based on comparison learning and label information provided by an embodiment of the application is shown in the figure.

[0046] Figure 3 A functional module diagram of a text classification device based on comparison learning and label information provided by an embodiment of the application is shown in the figure.

[0047] Figure 4 A structural diagram of an electronic device for implementing the text classification method based on comparison learning and label information provided by an embodiment of the application is shown in the figure.

[0048] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0050] The embodiment of the present application provides a text classification method based on comparison learning and label information. The execution subject of the text classification method based on comparison learning and label information includes but is not limited to at least one of electronic devices capable of being configured to execute the method provided by the embodiment of the present application, such as a server and a terminal. In other words, the text classification method based on comparison learning and label information can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server can be a stand-alone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0051] Reference Figure 1 As shown in the figure, the embodiment of the present application provides a flowchart of a text classification method based on comparison learning and label information. In this embodiment, the text classification method based on comparison learning and label information includes:

[0052] S1, obtaining a training text set with pre-labeled text labels, and converting each training text in the training text set into a text vector and converting the text label corresponding to each training text into a label vector by using a pre-constructed text classification model;

[0053] In the embodiment of the present application, medical text classification in the field of smart medical care is taken as an example to illustrate the text classification method based on comparison learning and label information. The training text set can be a medical record text after authorization by a patient, a medical literature text, or a medical instruction manual text, etc. The training text set can be obtained from a pre-constructed storage area after authorization, and can be grabbed by a computer statement (java statement, python statement, etc.) with data grabbing function. The pre-constructed storage area includes but is not limited to a database, a blockchain, and a network cache.

[0054] In the embodiment of the present application, each training text in the training text set is pre-labeled with a text label. The text label is a description information representing text classification type with a preset number of different types for a specific medical text classification scenario. For example, in lesion classification, the text label includes but is not limited to target organ classification labels such as kidney, spleen, left lung, and right lung, and in disease classification, the text label includes but is not limited to respiratory diseases, digestive diseases, and ophthalmic diseases.

[0055] In the embodiment of the present application, one training text in the training text set is labeled with only one text label, and the training text and the text label correspond to each other.

[0056] In the embodiment of the present application, the pre-constructed text classification model can be any type of model based on deep learning theory, for example, the pre-constructed text classification model can be constructed based on nnUNet convolutional neural network, or can be constructed based on Transformer model.

[0057] It can be understood that the training text set and the corresponding text label are texts composed of natural language, and if they are directly analyzed, a large amount of computing resources will be occupied, resulting in low efficiency of analysis, therefore, the pre-constructed text classification model can be used to convert each training text and each text label into a vector.

[0058] In detail, referring to Figure 2 As shown in the figure, the pre-constructed text classification model is used to convert each training text in the training text set into a text vector in turn, comprising:

[0059] S11, performing word segmentation processing on each training text in the training text set to obtain a word segmentation set corresponding to each training text;

[0060] S12, selecting one word segmentation from the word segmentation set as a target word segmentation, and counting the co-occurrence times of the target word segmentation and the adjacent word segmentation of the target word segmentation within the preset neighborhood range of the target word segmentation;

[0061] S13, constructing a co-occurrence matrix using the co-occurrence times corresponding to each word segmentation;

[0062] S14, converting the word segmentation corresponding to each training text into a word vector using the pre-constructed text classification model, and concatenating the word vectors into a vector matrix;

[0063] S15, performing product operation on the co-occurrence matrix and the vector matrix to obtain a text vector corresponding to each training text.

[0064] In the embodiment of the present application, a preset standard dictionary can be used to perform word segmentation processing on each training text to obtain a plurality of word segmentations, and the standard dictionary contains a plurality of standard word segmentations.

[0065] For example, the training text is searched in the standard dictionary according to different lengths, if the same standard word segmentation as the training text can be searched, it can be determined that the searched standard word segmentation is the word segmentation of the training text.

[0066] In another embodiment of the present application, a used word segmentation tool, for example, jieba word segmentation tool can be used to perform word segmentation processing on each training text.

[0067] Exemplarily, the co-occurrence matrix shown below can be constructed by using the co-occurrence number corresponding to each word segmentation:

[0068]

[0069] wherein X i,j is the co-occurrence number of word segmentation i and the adjacent word segmentation j of the word segmentation i in the training text.

[0070] In an optional embodiment of the present application, the pre-constructed text classification model can be a word2vec model, an NLP (Natural Language Processing) model or other models having a word vector conversion function, which converts multiple word segmentations into word vectors, and then concatenates the word vectors into a vector matrix of the training text.

[0071] It should be noted that the method of converting the text label corresponding to each training text into a label vector is the same as the method of converting each training text in the training text set into a text vector by using the pre-constructed text classification model, which will not be described here.

[0072] The embodiments of the present application convert the training text and the text label into vectors, so as to subsequently compare the training text and the text label based on the text vector of each training text and the label vector of each text label, and compare the text labels.

[0073] S2, sequentially calculate the first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text, and calculate the second loss value between each label vector, fuse the first loss value and the second loss value, and obtain a comprehensive loss value;

[0074] It can be understood that each text label contains a specific text semantic in addition to the role of classification index, for example, one text classification label is kidney, and another text classification label is respiratory department, both of which have certain semantics, and the text meanings expressed by the two text classification labels are different. At the same time, each training text also has certain semantic information, for example, a certain training text is a text describing the symptom of respiratory difficulty of a patient. In the actual classification training process of the pre-constructed text classification model, the final classification training result is to make each training text closer to the corresponding text label and farther away from other text labels. Therefore, we can adjust the uniform distribution of the training text and the text label in the semantic space by calculating the first loss value between each text vector and each label vector.

[0075] In detail, the first loss value between each text vector of the training text and a label vector corresponding to the training text is calculated in sequence, including:

[0076] The first loss value between each text vector and each label vector is calculated by using a first loss function as follows:

[0077]

[0078] Wherein, Loss1 represents the first loss value, h i represents the text vector of the i-th text in the training text set, l i represents the label vector of the text label corresponding to the i-th text, sim(h i , l i ) represents the cosine similarity between h i and l i , l j represents the label vector of the j-th text label corresponding to the text label, N represents the number of text labels in the preset text label set, sim(h i , l i ) represents the cosine similarity between h i and l j , and τ is a temperature coefficient.

[0079] It can be understood that the temperature coefficient adjusts the attention degree of the pre-constructed text classification model to the abnormal training text, and the smaller the temperature coefficient is, the more the pre-constructed text classification model pays attention to separating the training text from the most similar abnormal training text.

[0080] In the embodiment of the application, considering that each text label itself has certain semantic information, and there may be semantic similarity between multiple text labels, which may cause the boundary of the final text classification to be blurred. For example, one text label is left lung, and another text label is upper lung lobe. In order to improve the uniformity of the classification of the pre-constructed text classification model, each text label can be uniformly distributed in the semantic space by using the second loss value.

[0081] In detail, the second loss value between each label vector is calculated, including:

[0082] The second loss value between each label vector is calculated by using a second loss function as follows:

[0083]

[0084] wherein, Loss2 represents the second loss value, i and j represent the i-th text label and the j-th text label in the preset text label set respectively, i is not equal to j, l j represents a label vector of the j-th text label, l i represents a label vector of the i-th text label, sim(l i ,l j represents a cosine similarity between l i and l j .

[0085] In detail, the first loss value and the second loss value are fused to obtain a comprehensive loss value, comprising:

[0086] A preset first weight corresponding to the first loss value and a second weight corresponding to the second loss value are obtained.

[0087] The first loss value and the second loss value are weighted and averaged by using the first weight and the second weight to obtain the comprehensive loss value.

[0088] In the embodiment of the application, the first loss value and the second loss value are fused to obtain the comprehensive loss value, so that the candidate uses the comprehensive loss value to adjust the parameters of the pre-constructed text classification model, so that the text classification model meets the expected training requirements.

[0089] S3, the parameters of the pre-constructed text classification model are adjusted by using the comprehensive loss value, and the text classification model after parameter adjustment is iteratively trained until the comprehensive loss value meets the preset training condition, to obtain a trained text classification model and a nearest label vector of each text label.

[0090] In an optional embodiment of the application, the comprehensive loss value can be input into the gradient inversion layer of the pre-constructed text classification model to realize the inversion of the gradient, so as to adjust the training task of the pre-constructed text classification model.

[0091] In the embodiment of the application, the preset training condition can be that when the comprehensive loss value is less than or equal to a preset loss threshold, the training of the text classification model is stopped.

[0092] In another optional embodiment of the application, the training of the text classification model can also be controlled by limiting the number of training iterations of the text classification model, for example, when the number of iterative training of the text classification model reaches a preset maximum number of iterations, the training of the text classification model is stopped.

[0093] It can be understood that when the pre-constructed text classification model stops training, the corresponding text classification model outputs the label vector of each text label in the preset text label set in the last iteration training process, that is, each text label has the latest label vector.

[0094] The embodiment of the application adjusts the parameters of the pre-constructed text classification model by using the corresponding comprehensive loss value of the first loss value and the second loss value, trains the pre-constructed text classification model, and enables the model to sufficiently learn the meaning of the training text and the meaning of the text label.

[0095] S4, obtaining a text to be classified, and classifying the text to be classified according to the latest label vector of each text label by using the trained text classification model.

[0096] In the embodiment of the application, the trained text classification model is used to classify the text to be classified.

[0097] In detail, the classification of the text to be classified according to the latest label vector of each text label by using the trained text classification model includes:

[0098] Converting the text to be classified into a text vector to be classified by using the trained text classification model;

[0099] Calculating the probability value between the text vector to be classified and the latest label vector of each text label;

[0100] Selecting the text label corresponding to the probability value satisfying the preset classification condition as the classification label of the text to be classified.

[0101] In detail, the calculation of the probability value between the text vector to be classified and the latest label vector of each text label includes:

[0102] Extracting the text features of the text to be classified from the text vector to be classified, and extracting the label features of the corresponding text label from the latest label vector of each text label;

[0103] Calculating the probability value between the text features and each label feature by using a preset activation function.

[0104] In the embodiment of the application, the following preset activation function can be used:

[0105]

[0106] Wherein, p(a|x) is the relative probability between the text vector to be classified x and the text label a, w ais a weight vector of text label a, T is a transpose operator symbol, exp is an expectation operator symbol, and X is a preset number of text labels.

[0107] In another optional embodiment of the present application, the similarity between the to-be-classified text vector and each of the text label vectors closest to the to-be-classified text vector can be calculated, for example, the cosine similarity between the two can be calculated, and the text label with the largest similarity can be selected as the classification label of the to-be-classified text.

[0108] The present application drives the alignment of the training text and the corresponding text label in the semantic space by calculating a first loss value between the text vector of each training text in a training text set composed of medical text information and the label vector of the medical text label corresponding to the training text, at the same time, calculates a second loss value between each label vector to avoid the semantic similarity of the text labels, thereby causing the problem of blurred boundaries of text classification, finally adjusts the parameters of the pre-constructed text classification model by using the comprehensive loss value corresponding to the first loss value and the second loss value, trains the pre-constructed text classification model, so that the model can learn the meaning of the training text and the meaning of the text label, and then uses the trained text classification model to classify the to-be-classified text, which can improve the accuracy of medical text classification.

[0109] As Figure 3 shown is a functional module diagram of a text classification device based on alignment learning and label information according to an embodiment of the present application.

[0110] The text classification device based on alignment learning and label information 100 can be installed in an electronic device. According to the functions implemented, the text classification device based on alignment learning and label information 100 includes a text and label vector representation module 101, a text and label loss calculation module 102, a classification model training module 103, and a to-be-classified text classification module 104. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, and are stored in the memory of the electronic device.

[0111] In the present embodiment, the functions of each module / unit are as follows:

[0112] The text and label vector representation module 101 is configured to obtain a training text set with pre-labeled text labels, convert each text in the training text set into a text vector and each text label in the preset text label set into a label vector using a pre-constructed text classification model;

[0113] The text and label loss calculation module 102 is configured to sequentially calculate a first loss value between a text vector of each training text and a label vector of a text label corresponding to the training text, and calculate a second loss value between each label vector, and fuse the first loss value and the second loss value to obtain a comprehensive loss value.

[0114] The classification model training module 103 is configured to adjust parameters of the pre-constructed text classification model by using the comprehensive loss value, and iteratively train the text classification model after the parameter adjustment until the comprehensive loss value meets a preset training condition, to obtain a trained text classification model and a latest label vector of each text label.

[0115] The text classification module 104 is configured to obtain a text to be classified, and classify the text to be classified according to the latest label vector of each text label by using the trained text classification model.

[0116] In detail, the specific implementation of each module of the text classification device 100 based on comparison learning and label information is as follows:

[0117] Step one, obtain a training text set with pre-labeled text labels, and convert each training text in the training text set into a text vector and convert a text label corresponding to each training text into a label vector by using a pre-constructed text classification model.

[0118] In the embodiment of the present application, medical text classification in the field of smart medical care is taken as an example to illustrate the text classification method based on comparison learning and label information. The training text set can be a medical record text after authorization by a patient, a medical literature text, a medical instruction manual text, etc. The training text set can be obtained from a pre-constructed storage area after authorization, can be grabbed by using a computer statement (java statement, python statement, etc.) with data grabbing function, and the pre-constructed storage area includes but is not limited to a database, a blockchain, and a network cache.

[0119] In the embodiment of the present application, each training text in the training text set is pre-labeled with a text label, and the text label is a preset number of description information representing text classification types for a specific medical text classification scenario. For example, in lesion classification, the preset text label set includes but is not limited to target organ classification labels such as kidney, spleen, left lung, and right lung, and in disease classification, the preset text label set includes but is not limited to respiratory diseases, digestive diseases, and ophthalmic diseases.

[0120] In the embodiment of the present application, one training text in the training text set is labeled with only one text label, and the training text corresponds to the text label one by one.

[0121] In the embodiment of the present application, the pre-constructed text classification model can be any type of model based on deep learning theory, for example, the pre-constructed text classification model can be constructed based on nnUNet convolutional neural network, or can be constructed based on Transformer model.

[0122] It can be understood that the training text set and the corresponding text label set are texts composed of natural language, and if they are directly analyzed, a large amount of computing resources will be occupied, resulting in low efficiency of analysis, therefore, the pre-constructed text classification model can be used to convert each training text and each text label into a vector.

[0123] In detail, the pre-constructed text classification model is used to convert each training text in the training text set into a text vector in turn, comprising:

[0124] Each training text in the training text set is subjected to word segmentation processing to obtain a word segmentation set corresponding to each training text;

[0125] One word segmentation in the word segmentation set is selected as a target word segmentation, and the co-occurrence number of the target word segmentation and the adjacent word segmentation of the target word segmentation within the preset neighborhood range of the target word segmentation is counted;

[0126] A co-occurrence matrix is constructed using the co-occurrence number corresponding to each word segmentation;

[0127] Each word segmentation corresponding to each training text is converted into a word vector using the pre-constructed text classification model, and the word vectors are spliced into a vector matrix;

[0128] The co-occurrence matrix and the vector matrix are multiplied to obtain a text vector corresponding to each training text.

[0129] In the embodiment of the present application, a preset standard dictionary can be used to perform word segmentation processing on each training text to obtain a plurality of word segmentations, and the standard dictionary contains a plurality of standard word segmentations.

[0130] For example, the training text is searched in the standard dictionary according to different lengths, if the same standard word segmentation as the training text can be searched, it can be determined that the searched standard word segmentation is the word segmentation of the training text.

[0131] In another embodiment of the present application, a used word segmentation tool, for example, a jieba word segmentation tool, can be used to perform word segmentation processing on each training text.

[0132] Exemplarily, the co-occurrence matrix shown below can be constructed by using the co-occurrence number corresponding to each segmented word:

[0133]

[0134] wherein, X i,j is the co-occurrence number of segmented word i and adjacent segmented word j of the segmented word i in the training text.

[0135] In an optional embodiment of the present application, the pre-constructed text classification model can be a word2vec model, an NLP (Natural Language Processing) model or other models having a word vector conversion function, which converts a plurality of segmented words into word vectors, and then splices the word vectors into a vector matrix of the training text.

[0136] It should be noted that the method of converting the text label corresponding to each training text into a label vector is the same as the method of converting each training text in the training text set into a text vector by using the pre-constructed text classification model, which will not be described here.

[0137] The embodiments of the present application convert the training text and the text label into vectors, so as to subsequently compare the training text and the text label based on the text vector of each training text and the label vector of each text label, and compare the text labels.

[0138] Step two, sequentially calculate the first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text, and calculate the second loss value between each label vector, fuse the first loss value and the second loss value, and obtain a comprehensive loss value;

[0139] It can be understood that each text label contains a specific text semantic in addition to the role of classification index. In the actual classification training process of the pre-constructed text classification model, the final classification training result is to make each training text and the corresponding text label closer, while being far away from other text labels. Therefore, we can adjust the uniform distribution of the training text and the text label in the semantic space by calculating the first loss value between each text vector and each label vector.

[0140] In detail, the sequentially calculating the first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text comprises:

[0141] The first loss value between each of the text vectors and each of the label vectors is calculated by using a first loss function as follows:

[0142]

[0143] wherein Loss1 represents the first loss value, h i represents a text vector of an i-th text in the training text set, l i represents a label vector of a text label corresponding to the i-th text, sim(h i , l i ) represents a cosine similarity between h i and l i , l j represents a label vector of a j-th text label, and N represents a number of text labels in the preset text label set, sim(h i , l i ) represents a cosine similarity between h i and l j , and τ is a temperature coefficient.

[0144] It can be understood that the temperature coefficient adjusts the attention degree of the pre-constructed text classification model to the abnormal training text, and the smaller the temperature coefficient is, the more the pre-constructed text classification model pays attention to separating the training text from the most similar abnormal training text.

[0145] In the embodiment of the present application, considering that each of the text labels has certain semantic information itself, and there may be semantic similarity between multiple text labels, which may cause the boundary of the final text classification to be blurred. In order to improve the uniformity of the classification of the pre-constructed text classification model, each of the text labels can be uniformly distributed in the semantic space by using the second loss value.

[0146] In detail, the second loss value between each of the label vectors is calculated, comprising:

[0147] The second loss value between each of the label vectors is calculated by using a second loss function as follows:

[0148]

[0149] wherein Loss2 represents the second loss value, i and j respectively represent an i-th text label and a j-th text label in the preset text label set, and i is not equal to j, l j represents a label vector of the j-th text label, l i represents a label vector of the i-th text label, sim(l i , l j) represents l i with l j a cosine similarity between l

[0150] In detail, the first loss value and the second loss value are fused to obtain a comprehensive loss value, including:

[0151] A preset first weight corresponding to the first loss value and a second weight corresponding to the second loss value are obtained.

[0152] The first loss value and the second loss value are weighted and averaged by using the first weight and the second weight to obtain the comprehensive loss value.

[0153] In the embodiment of the application, the first loss value and the second loss value are fused to obtain the comprehensive loss value, so that the candidate uses the comprehensive loss value to adjust the parameters of the pre-constructed text classification model, so that the text classification model meets the expected training requirements.

[0154] Step three, the parameters of the pre-constructed text classification model are adjusted by using the comprehensive loss value, and the text classification model after parameter adjustment is iteratively trained until the comprehensive loss value meets the preset training condition, to obtain a trained text classification model and a nearest label vector of each text label;

[0155] In an optional embodiment of the application, the comprehensive loss value can be input into the gradient inversion layer of the pre-constructed text classification model to realize gradient inversion, so as to adjust the training task of the pre-constructed text classification model.

[0156] In the embodiment of the application, the preset training condition can be that when the comprehensive loss value is less than or equal to a preset loss threshold, the training of the text classification model is stopped.

[0157] In another optional embodiment of the application, the training of the text classification model can also be controlled by limiting the number of training iterations of the text classification model, for example, when the number of iterative training of the text classification model reaches a preset maximum number of iterations, the training of the text classification model is stopped.

[0158] It can be understood that when the pre-constructed text classification model stops training, the corresponding text classification model will output the label vector of each text label in the preset text label set in the process of the last iterative training, that is, each text label has a nearest label vector.

[0159] The embodiment of the present application adjusts the parameters of the pre-constructed text classification model by using the comprehensive loss value corresponding to the first loss value and the second loss value, and trains the pre-constructed text classification model, so that the model can sufficiently learn the meaning of the training text and the meaning of the text label.

[0160] Step four, obtaining the text to be classified, and classifying the text to be classified according to the nearest label vector of each text label by using the trained text classification model.

[0161] In the embodiment of the present application, the trained text classification model is used to classify the text to be classified.

[0162] In detail, the classification of the text to be classified according to the nearest label vector of each text label by using the trained text classification model comprises:

[0163] Converting the text to be classified into a text vector to be classified by using the trained text classification model;

[0164] Calculating the probability value between the text vector to be classified and the nearest label vector of each text label;

[0165] Selecting the text label corresponding to the probability value satisfying the preset classification condition as the classification label of the text to be classified.

[0166] In detail, the calculation of the probability value between the text vector to be classified and the nearest label vector of each text label comprises:

[0167] Extracting the text features of the text to be classified from the text vector to be classified, and extracting the label features of the corresponding text label from each nearest label vector of the text label;

[0168] Calculating the probability value between the text features and each label feature by using a preset activation function.

[0169] In the embodiment of the present application, the following preset activation function can be used:

[0170]

[0171] Wherein, p(a|x) is the relative probability between the text vector to be classified x and the text label a, w a is the weight vector of the text label a, T is the transpose operator symbol, exp is the expectation operator symbol, and X is the number of preset text labels.

[0172] In another optional embodiment of the present application, the similarity between the to-be-classified text vector and each of the text label vectors closest to the to-be-classified text vector is calculated, for example, the cosine similarity between the two is calculated, and the text label with the largest similarity is selected as the classification label of the to-be-classified text.

[0173] In the present application, the first loss value between the text vector of each training text in the training text set and the label vector of the corresponding text label is calculated to drive the alignment of the training text and the corresponding text label in the semantic space, and the second loss value between each label vector is calculated to avoid the semantic similarity of the text labels, thereby solving the problem of blurred boundaries of text classification. Finally, the parameters of the pre-constructed text classification model are adjusted by using the comprehensive loss value corresponding to the first loss value and the second loss value, and the pre-constructed text classification model is trained, so that the model can sufficiently learn the meaning of the training text and the meaning of the text label, and then the trained text classification model is used for classification of the to-be-classified text, which can improve the accuracy of text classification.

[0174] As shown in Figure 4 FIG. 1 is a structural schematic diagram of an electronic device for implementing the text classification method based on alignment learning and label information according to an embodiment of the present application.

[0175] The electronic device 1 can include a processor 10, a memory 11 and a bus, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as text classification based on alignment learning and label information.

[0176] The memory 11 includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 1. Further, the memory 11 can include both the internal storage unit and the external storage device of the electronic device 1. The memory 11 can be used not only to store application software and various data installed on the electronic device 1, such as the code of the text classification based on alignment learning and label information, but also to temporarily store data that has been output or will be output.

[0177] The processor 10 can be composed of integrated circuits in some embodiments, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the memory 11 (such as text classification based on comparison learning and label information, etc.), and calls data stored in the memory 11 to perform various functions and process data of the electronic device 1.

[0178] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize the connection and communication between the memory 11, the at least one processor 10, etc.

[0179] Figure 4 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 4 The structure shown does not constitute a limitation on the electronic device 1, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0180] For example, although not shown, the electronic device 1 can also include a power supply (such as a battery) for powering various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so as to realize functions such as charge management, discharge management, and power consumption management through the power management device. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which are not described here.

[0181] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is usually used to establish a communication connection between the electronic device 1 and other electronic devices.

[0182] Optionally, the electronic device 1 can further include a user interface, which can be a display, an input unit such as a keyboard, and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, for displaying information processed in the electronic device 1 and for displaying a visualized user interface.

[0183] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by the structure.

[0184] The text classification based on the comparison learning and the label information stored in the memory 11 in the electronic device 1 is a combination of a plurality of instructions, which, when executed in the processor 10, can achieve:

[0185] Obtain a training text set with pre-labeled text labels, and convert each training text in the training text set into a text vector and each text label corresponding to the training text into a label vector using a pre-built text classification model;

[0186] Calculate a first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text, and a second loss value between each label vector, fuse the first loss value and the second loss value, and obtain a comprehensive loss value;

[0187] Adjust the parameters of the pre-built text classification model using the comprehensive loss value, and iteratively train the text classification model after adjusting the parameters until the comprehensive loss value meets a preset training condition, to obtain a trained text classification model and a recent label vector of each text label;

[0188] Obtain a text to be classified, and classify the text to be classified according to the recent label vector of each text label using the trained text classification model.

[0189] Further, the modules / units integrated in the electronic device 1, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0190] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device:

[0191] obtaining a training text set with pre-labeled text labels, converting each training text in the training text set into a text vector and each text label corresponding to the training text into a label vector in sequence by using a pre-built text classification model;

[0192] calculating a first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text, and a second loss value between each label vector in sequence, fusing the first loss value and the second loss value to obtain a comprehensive loss value;

[0193] adjusting parameters of the pre-built text classification model by using the comprehensive loss value, and iteratively training the text classification model after the parameter adjustment until the comprehensive loss value meets a preset training condition, to obtain a trained text classification model and a latest label vector of each text label;

[0194] obtaining a text to be classified, and classifying the text to be classified according to the latest label vector of each text label by using the trained text classification model.

[0195] In addition, each functional module in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software function module.

[0196] It is obvious for those skilled in the art that the application is not limited to the details of the above exemplary embodiments, and the application can be realized in other specific forms without departing from the spirit or essential characteristics of the application.

[0197] Thus, the embodiments can be regarded as illustrative rather than restrictive, and the scope of the application is defined by the appended claims rather than the description preceding it, so that all changes that come within the meaning and range of equivalency of the claims are to be embraced within the scope of the application. Any reference signs in the claims should not be construed as limiting the scope of the claims.

[0198] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. The blockchain is essentially a decentralized database, and is a series of data blocks associated using cryptographic methods. Each data block contains information of a batch of network transactions, and is used to verify the validity (anti-fake) of the information and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0199] Embodiments of the present application can acquire and process related data based on holographic projection technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0200] In addition, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. The second and subsequent words are used to indicate the name, not any particular order.

[0201] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A text classification method based on comparison learning and label information, characterized in that, The method comprises: acquiring a training text set with pre-labeled text labels, and converting each training text in the training text set into a text vector in sequence by using a pre-built text classification model, and converting a text label corresponding to each training text into a label vector; calculating a first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text in sequence, and calculating a second loss value between each label vector, fusing the first loss value and the second loss value to obtain a comprehensive loss value; inputting the comprehensive loss value into a gradient inversion layer of the pre-built text classification model to perform inversion on the gradient of the pre-built text classification model, adjusting the parameters of the pre-built text classification model, and iteratively training the text classification model after the parameter adjustment until the comprehensive loss value meets a preset training condition, thereby obtaining a trained text classification model and a latest label vector of each text label; acquiring a text to be classified, and classifying the text to be classified according to the latest label vector of each text label by using the trained text classification model. 2.The text classification method based on comparison learning and label information according to claim 1, wherein, The method comprises: performing word segmentation processing on each training text in the training text set to obtain a word segmentation set corresponding to each training text; selecting one word segmentation from the word segmentation set as a target word segmentation, and counting the co-occurrence times of the target word segmentation and adjacent word segmentations of the target word segmentation within a preset neighborhood range of the target word segmentation; constructing a co-occurrence matrix by using the co-occurrence times corresponding to each word segmentation; converting the word segmentations corresponding to each training text into word vectors respectively by using the pre-built text classification model, and concatenating the word vectors into a vector matrix; performing product operation on the co-occurrence matrix and the vector matrix to obtain a text vector corresponding to each training text. 3.The text classification method based on comparison learning and label information according to claim 1, wherein, The method comprises: calculating the first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text by using the following first loss function: wherein, denotes the first loss value, denotes a text vector of an i-th text in the training text set, denotes a label vector of a text label corresponding to the i-th text, denotes a cosine similarity between and denotes a label vector of a text label corresponding to the i-th text, denotes a cosine similarity between and denotes a label vector of a text label corresponding to the i-th text, denotes a number of text labels in a preset text label set, denotes a cosine similarity between and denotes a cosine similarity between and is a temperature coefficient. 4.The text classification method based on comparison learning and label information according to claim 1, wherein, The method comprises: calculating the second loss value between each label vector by using the following second loss function: in, This represents the second loss value. and These represent the first and second tags in the preset text tag set. The first text label and the first A text label, and Not equal to , Indicates the first A tag vector for each text tag. Indicates the first A tag vector for each text tag. express and Cosine similarity between them. 5.The text classification method based on comparison learning and label information according to claim 1, wherein, The method comprises: obtaining a preset first weight corresponding to the first loss value and a second weight corresponding to the second loss value; performing weighted averaging on the first loss value and the second loss value by using the first weight and the second weight to obtain the comprehensive loss value. 6.The text classification method based on comparison learning and label information according to claim 1, wherein, The method comprises: converting the text to be classified into a text vector to be classified by using the trained text classification model; Calculate the probability value between the text vector to be classified and the nearest tag vector for each text tag; The text labels corresponding to the probability values ​​that meet the preset classification conditions are selected as the classification labels of the text to be classified. 7.The text classification method based on comparison learning and label information according to claim 6, wherein, The calculation of the probability value between the text vector to be classified and the nearest tag vector for each text tag includes: Calculate the cosine similarity between the text vector to be classified and the nearest tag vector of each text tag in turn; All cosine similarities are normalized, and the normalized cosine similarities are converted into the probability values.

8. A text classification apparatus based on comparison learning and label information, characterized by, The device includes: The text and label vector representation module is used to obtain a training text set with pre-labeled text labels, and to use a pre-built text classification model to convert each training text in the training text set into a text vector, and to convert the text label corresponding to each training text into a label vector. The text and label loss calculation module is used to sequentially calculate a first loss value between the text vector of each training text and the label vector of the text label corresponding to the training text, and calculate a second loss value between each label vector, and fuse the first loss value and the second loss value to obtain a comprehensive loss value; The classification model training module is used to input the comprehensive loss value into the gradient inversion layer of the pre-built text classification model to invert the gradient of the pre-built text classification model, adjust the parameters of the pre-built text classification model, and iteratively train the text classification model with adjusted parameters until the comprehensive loss value meets the preset training conditions, thereby obtaining the trained text classification model and the nearest label vector for each text label. The text classification module is used to obtain the text to be classified and classify the text according to the nearest label vector of each text label using the trained text classification model.

9. An electronic device, comprising: The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the text classification method based on comparison learning and label information as described in any one of claims 1 to 7.

10. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the text classification method based on comparison learning and label information as described in any one of claims 1 to 7.

Images