Text classification label correction method, device, equipment and storage medium

Abstract

The present application provides a method, device, equipment and storage medium for correcting text classification labels. The method includes: calculating the first classification support rate of the text to be corrected in each set classification category based on the classification results of the text to be corrected in each trained text classifier; predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected; and correcting the marked classification labels of the text to be corrected using the first classification support rate and the second classification support rate of the text to be corrected in each classification category. The present application analyzes the actual classification of the text to be corrected from the two perspectives of the text classifier and the associated vocabulary set, realizes the correction of the text classification label, ensures the accuracy of the text classification label, and subsequently uses such corrected text to train the text classification model to improve the classification accuracy of the text classification model.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically to a method, device, equipment and storage medium for correcting text classification labels. Background Art

[0002] With the rapid development of deep learning technology, pre-trained text classification models are used to classify various types of text, which significantly improves the text classification effect. At this time, when training the corresponding text classification model, a large amount of text data with classification labels is required as training samples for the text classification model.

[0003] At present, considering that it is expensive and time-consuming for experts to mark the classification labels of each text, for each text used as a training sample, the classification labels for each text are usually collected from crowdsourcing by public volunteers. However, the text classification labels collected from crowdsourcing are affected by various annotation noises. For example, compared with experts, public volunteers have weaker understanding of each text, and there are problems with labeling errors, which makes the text classification labels of the training samples inaccurate, and further leads to low classification accuracy of the trained text classification model for various types of text. Summary of the invention

[0004] The present application provides a text classification label correction method, device, equipment and storage medium, which analyzes the actual classification of each text from two perspectives to correct the classification label of each text, ensure the accuracy of the text classification label, and further improve the classification accuracy of the text classification model trained using such text.

[0005] In a first aspect, an embodiment of the present application provides a method for correcting a text classification label, the method comprising:

[0006] Based on the classification results of the text to be corrected in each trained text classifier, calculating the first classification support rate of the text to be corrected in each set classification category;

[0007] Predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected;

[0008] The classification labels marked on the text to be corrected are corrected by using the first classification support rate and the second classification support rate of the text to be corrected in each classification category.

[0009] Furthermore, the first classification support rate of the text to be corrected in each set classification category is calculated based on the classification result of the text to be corrected in each trained text classifier, including:

[0010] Inputting the text to be corrected into each trained text classifier respectively to obtain a classification result matrix of the text to be corrected;

[0011] Based on the occurrence frequency of the category name of each classification category in the classification result matrix, the first classification support rate of the text to be corrected under the classification category is calculated.

[0012] Furthermore, before calculating the first classification support rate of the text to be corrected in each set classification category based on the classification result of the text to be corrected in each trained text classifier, the method further includes:

[0013] Extract the corresponding test text sets in batches from the constructed text training library;

[0014] Using the test text sets extracted from each batch, respectively, to train the corresponding text classifiers to obtain the trained text classifiers;

[0015] The test text sets extracted in different batches are different.

[0016] Furthermore, the predicting of the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected includes:

[0017] Counting the number of occurrences of the associated words in the associated word set under each classification category in the text to be corrected;

[0018] The second classification support rate of the text to be corrected in each classification category is calculated according to the number of occurrences in each classification category and the total number of words in the text to be corrected.

[0019] Furthermore, before predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected, the method further includes:

[0020] The replaceable words of the category name of each classification category are analyzed to construct a set of related words under the classification category.

[0021] Furthermore, the analysis of replaceable words of the category name of each classification category includes:

[0022] For each classification category, a target text under the classification category is searched from the constructed text training library, wherein the target text contains the category name of the classification category;

[0023] The category name of the classification category is hidden in each of the target texts, and each of the hidden target texts is input into the constructed semantic replacement model to output replaceable words for the category name of the classification category.

[0024] Furthermore, the method of correcting the marked classification labels of the text to be corrected by using the first classification support rate and the second classification support rate of the text to be corrected in each classification category includes:

[0025] respectively determining a first category to which the to-be-corrected text has a maximum support rate within the first classification, and a second category to which the to-be-corrected text has a maximum support rate within the second classification;

[0026] If the first category and the second category are the same, the category label annotated on the text to be corrected is corrected to the category name of the same category.

[0027] Furthermore, the number of the trained text classifiers is greater than or equal to the number of set classification categories, and the text to be corrected is any text in the constructed text training library.

[0028] In a second aspect, an embodiment of the present application provides a device for correcting text classification labels, the device comprising:

[0029] A text classification calculation module, used to calculate the first classification support rate of the text to be corrected in each set classification category based on the classification result of the text to be corrected in each trained text classifier;

[0030] A text classification prediction module, used for predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected;

[0031] The classification label correction module is used to correct the classification labels marked on the text to be corrected by using the first classification support rate and the second classification support rate of the text to be corrected in each classification category.

[0032] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0033] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the text classification label correction method provided in the first aspect of the present application.

[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the text classification label correction method provided in the first aspect of the present application.

[0035] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the method for correcting text classification labels as provided in the first aspect of the present application is implemented.

[0036] The embodiments of the present application provide a method, apparatus, device and storage medium for correcting text classification labels. A plurality of text classifiers are pre-trained, and a corresponding associated vocabulary set is constructed in advance for each set classification category. Thus, when correcting the classification label of the text to be corrected, the first classification support rate of the text to be corrected in each classification category is calculated through the classification result of the text to be corrected in each text classifier, and the second classification support rate of the text to be corrected in each classification category is predicted through the association frequency of the associated vocabulary set in each classification category in the text to be corrected. Thus, the actual classification of the text to be corrected is analyzed from the two perspectives of the text classifier and the associated vocabulary set, so as to correct the marked classification labels of the text to be corrected to ensure the accuracy of the text classification labels. Subsequently, such corrected text can be used to train the corresponding text classification model, thereby improving the classification accuracy of the trained text classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A flowchart of a text classification label correction method shown in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of the principle of the text classification label correction process shown in an embodiment of the present application;

[0040] Figure 3 A flowchart of another method for correcting text classification labels shown in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of the principle of the training process of the text classifier shown in the embodiment of the present application;

[0042] Figure 5A schematic diagram of the principle of the construction process of the associated vocabulary set under each classification category shown in the embodiment of the present application;

[0043] Figure 6 A principle block diagram of a text classification label correction device shown in an embodiment of the present application;

[0044] Figure 7 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0047] Taking into account that when training a text classification model, there may be labeling errors in the classification labels marked in a large amount of text obtained from crowdsourcing, which may lead to the problem of low classification accuracy of the text classification model, the embodiment of the present application designs a method for correcting the classification labels of each text in the crowdsourcing, and analyzes the actual classification status of each text in the crowdsourcing from two different angles to correct the classification labels of each text, thereby ensuring the accuracy of the text classification labels, and thereby improving the classification accuracy of the text classification model trained using such text.

[0048] Figure 1 This is a flowchart of a method for correcting text classification labels shown in an embodiment of the present application. Figure 1 , the method may specifically include the following steps:

[0049] S110, based on the classification results of the text to be corrected in each trained text classifier, calculating the first classification support rate of the text to be corrected in each set classification category.

[0050] Considering that when training a text classification model for determining the classification category to which each text belongs, each type of text as training sample has been annotated with corresponding classification labels, but in order to ensure the classification accuracy of the trained text classification model, it is required that the annotated classification labels of each type of text as training samples can also maintain a certain accuracy. Therefore, after obtaining a large number of various types of text annotated with corresponding classification labels from crowdsourcing, it is necessary to accurately calibrate the classification labels annotated for each text.

[0051] For the training of text classification models, a large amount of text obtained from crowdsourcing will be used to build a corresponding text training library. It can be seen that the text to be corrected in this application is any text in the constructed text training library, so that the classification label of each text in the text training library can be corrected to ensure the accuracy of the text classification label.

[0052] At this time, in order to accurately determine the true classification of each text, the present application can analyze the category to which the text to be corrected belongs from two different perspectives: model classification and text semantics, so as to comprehensively analyze the actual classification type of the text to be corrected, thereby accurately correcting the classification labels that have been annotated with the text to be corrected. In this step, the method of analyzing the classification results of the text to be corrected from the perspective of model classification is mainly described in detail.

[0053] Considering that when the corresponding text classifier is trained using each text with uncorrected classification labels, the classification accuracy of the text classifier for each type of text is relatively low, the present application can establish multiple different training sample sets, and different training sample sets contain a large number of texts with different contents. For example, sets consisting of texts with different contents can be selected from the constructed text training library as each training sample set.

[0054] Then, a corresponding text classifier can be trained using each text in each training sample set, thereby obtaining multiple text classifiers with low classification accuracy. At this time, since the training sample sets used by each text classifier are different, the classification performance of each text classifier also has certain differences, so the classification results of each text classifier when classifying the same text may also have certain differences.

[0055] Moreover, in order to ensure the accuracy of text classification by the text classifier to a certain extent, the present application will set up a variety of different classification categories in advance for the categories to which various types of text may belong. Each classification category is also a classification category set in the text classifier, so that the text classifier can accurately determine to a certain extent the category to which a certain text actually belongs in each classification category.

[0056] As an optional implementation scheme in the embodiment of the present application, when correcting a certain text with annotated classification label, the text to be corrected is first input into each trained text classifier, such as Figure 2 As shown, each text classifier performs a corresponding analysis on the category to which the text to be corrected belongs, thereby obtaining the classification result of each text classifier for the text to be corrected. At this time, the classification results of each text classifier for the same text to be corrected may be different. Then, by comprehensively analyzing the classification results of each text classifier for the text to be corrected, the possibility of the text to be corrected belonging to each classification category is determined, thereby calculating the first classification support rate of the text to be corrected in each classification category.

[0057] It should be noted that, considering that the classification results of each text classifier for the same text may include all classification categories, in order to ensure the comprehensive accuracy of the first classification support rate, this application requires that the number of trained text classifiers is greater than or equal to the number of set classification categories, so that the classification results of each text classifier for the same text can comprehensively include all possible classification categories.

[0058] S120: predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected.

[0059] In this step, the method of analyzing the classification results of the text to be corrected from the perspective of text semantics is mainly explained in detail.

[0060] Specifically, considering that the semantics of different texts are usually determined by the various words contained in the text, the present application can analyze the semantic category of the text to be corrected by judging the number of words related to each classification category contained in the text to be corrected. For example, if most of the words in the text to be corrected are related to the "sports" category, then it can be determined that the text to be corrected belongs to the "sports" category.

[0061] As an optional implementation scheme in the embodiment of the present application, in order to accurately determine the number of words related to each classification category contained in the text to be corrected, a corresponding associated vocabulary set is constructed in advance for each classification category. The associated vocabulary set under each classification category contains a variety of words that have a certain association relationship with the classification category. For example, the associated vocabulary set under the "sports" category may include various sports-related words such as "sports", "football", "basketball", and "running".

[0062] In the present application, when determining the number of words related to each classification category contained in the text to be corrected, it is possible to analyze for each classification category whether the text to be corrected contains each associated word contained in the associated word set under the classification category, such as Figure 2 As shown. Then, according to the number of such related words in the text to be corrected, the number of words related to the classification category contained in the text to be corrected is determined, so as to analyze the possibility that the text to be corrected belongs to the classification category, thereby predicting the second classification support rate of the text to be corrected under the classification category. The above process is performed for each classification category, and the second classification support rate of the text to be corrected under each classification category can be predicted.

[0063] It should be understood that the first category support rate and the second category support rate of the text to be corrected in each classification category are respectively used to indicate the possibility that the text to be corrected belongs to each classification category analyzed from two different perspectives, model classification and text semantics. Subsequently, by comprehensively analyzing the first category support rate and the second category support rate, the true category to which the text to be corrected belongs can be accurately determined.

[0064] It should be noted that there is no specific execution order between S110 and S120 in the present application. They can be executed simultaneously or sequentially, and there is no limitation on this.

[0065] S130, using the first classification support rate and the second classification support rate of the text to be corrected in each classification category, correct the classification labels that have been marked on the text to be corrected.

[0066] After obtaining the first classification support rate and the second classification support rate of the text to be corrected in each classification category, the classification category to which the text to be corrected is most likely to belong can be determined from the perspective of model classification according to the size of the first classification support rate of the text to be corrected in each classification category. Moreover, the classification category to which the text to be corrected is most likely to belong can also be determined from the perspective of text semantics according to the size of the second classification support rate of the text to be corrected in each classification category. Then, by judging whether the two classification categories to which the text to be corrected is most likely to belong determined from the two perspectives of model classification and text semantics are consistent, it can be determined whether the classification label that has been marked for the text to be corrected is wrong, so that the classification label that has been marked for the text to be corrected can be accurately corrected to ensure the accuracy of the text classification label.

[0067] Exemplarily, the steps of correcting the marked classification labels of the text to be corrected in the present application can be specifically divided into: determining the first category to which the text to be corrected has the maximum support rate within the first category, and the second category to which the text to be corrected has the maximum support rate within the second category; if the first category and the second category are the same, correcting the marked classification labels of the text to be corrected to the category name of the same category.

[0068] That is to say, the maximum value within the first classification support rate can be found from the first classification support rate of the text to be corrected under each classification category, and the first category corresponding to the maximum value can represent the classification category to which the text to be corrected is most likely to belong, determined from the perspective of model classification. In addition, the maximum value within the second classification support rate can be found from the second classification support rate of the text to be corrected under each classification category, and the second category corresponding to the maximum value can represent the classification category to which the text to be corrected is most likely to belong, determined from the perspective of text semantics. If the first category and the second category are the same, it means that the text to be corrected is determined to belong to the same category from two different perspectives, and the same category can be used as the true classification category of the text to be corrected. Therefore, when the classification label of the text to be corrected is the same as the category name of the same category, it means that the classification label of the text to be corrected is accurately marked and does not need to be corrected; and when the classification label of the text to be corrected is different from the category name of the same category, it means that the classification label of the text to be corrected is incorrectly marked, and the classification label of the text to be corrected can be corrected to the category name of the same category to ensure the accuracy of the text classification label.

[0069] In addition, if the first category and the second category are different, it means that the real classification category of the text to be corrected cannot be accurately determined from two different angles. In this case, the classification label already marked on the text to be corrected shall prevail, and no correction operation shall be performed on the classification label already marked on the correction text.

[0070] It should be understood that each text in the constructed text training library can be used as the text to be corrected in this application. By executing the above-mentioned text classification label correction step, the classification label of each text in the text training library can be accurately corrected, thereby ensuring the accuracy of each text classification label in the text training library. When the text training library is subsequently used to train the corresponding text classification model, the classification accuracy of the text classification model can be further improved.

[0071] The technical solution provided in the embodiment of the present application pre-trains multiple text classifiers and constructs a corresponding associated vocabulary set for each set classification category in advance, so that when the classification label of the text to be corrected is corrected, the first classification support rate of the text to be corrected in each classification category is calculated through the classification result of the text to be corrected in each text classifier, and the second classification support rate of the text to be corrected in each classification category is predicted through the association frequency of the associated vocabulary set under each classification category in the text to be corrected, so as to analyze the actual classification situation of the text to be corrected from the two perspectives of the text classifier and the associated vocabulary set, so as to correct the marked classification label of the text to be corrected to ensure the accuracy of the text classification label, and subsequently such corrected text can be used to train the corresponding text classification model, thereby improving the classification accuracy of the trained text classification model.

[0072] As an optional implementation scheme in the embodiment of the present application, in order to ensure accurate correction of text classification labels, the present application describes in detail the specific determination process of the first classification support rate and the second classification support rate of the text to be corrected in each classification category.

[0073] Figure 3 A flowchart of another method for correcting text classification labels shown in an embodiment of the present application.

[0074] like Figure 3 As shown, the method may specifically include the following steps:

[0075] S310, input the text to be corrected into each trained text classifier respectively to obtain a classification result matrix of the text to be corrected.

[0076] When analyzing the possibility of the category to which the text to be corrected actually belongs from the perspective of model classification, the text to be corrected can be input into each trained text classifier, and each text classifier analyzes the category to which the text to be corrected belongs, and obtains the classification result output by each text classifier for the text to be corrected. Furthermore, the classification results output by each text classifier for the text to be corrected are combined to obtain the classification result matrix of the text to be corrected.

[0077] For example, the classification result matrix of the text to be corrected can be Y=[L,y j ,L] means, where y j Represents the category name of the j-th classification category, and there are n elements in the classification result matrix, where n is the number of text classifiers.

[0078] It should be noted that in the present application, the following steps can be performed for the training of each text classifier: extracting corresponding test text sets in batches from the constructed text training library; using the test text sets extracted from each batch to train the corresponding text classifier to obtain trained text classifiers.

[0079] That is to say, in order to obtain training samples for each text classifier, the present application can extract a corresponding number of texts in batches from a pre-built text training library, such as Figure 4 As shown, a test text set corresponding to each batch is formed. For example, 20% of the text data can be extracted from the text training library each time to form a corresponding test text set, and the remaining 80% of the text data can be used as a verification text set for the text classifier trained using the test text set, so as to subsequently verify the classification ability of the text classifier. Then, multiple test text sets can be obtained by batch extraction, and a corresponding text classifier can be trained using the test text sets extracted from each batch. After training after batch extraction, each trained text classifier can be obtained.

[0080] It should be understood that in order to ensure the comprehensiveness of the classification of the text to be corrected by multiple text classifiers, there will be certain differences between the classification performance of each text classifier. Therefore, the corresponding test text sets are extracted from the text training library in batches, and each batch will select texts with different contents for extraction, so that the test text sets extracted in different batches are different.

[0081] S320, calculating the first classification support rate of the text to be corrected under each classification category based on the occurrence frequency of the category name of each classification category in the classification result matrix.

[0082] After obtaining the classification result matrix of the text to be corrected, the classification result matrix will contain the classification results output by each text classifier for the text to be corrected. Therefore, when analyzing the possibility that the text to be corrected belongs to each classification category, the number of occurrences of the category name of the classification category in the classification result matrix (that is, the frequency of occurrence in this application) will be counted for each classification category, so as to calculate the proportion of the number of occurrences to the total number of classification results (that is, the number of trained text classifiers) contained in the classification result matrix as the first classification support rate of the text to be corrected in the classification category. The above process is performed for each classification category respectively, and the first classification support rate of the text to be corrected in each classification category can be calculated.

[0083] Exemplarily, the calculation formula for the first classification support rate of the text to be corrected in each classification category in the present application may be: Among them, yj is the category name of the j-th classification category, count() is the counting function, count(yj ) represents the number of occurrences in the classification result matrix when the classification result output by the text classifier is the category name of the j-th classification category, and n is the number of text classifiers.

[0084] S330, counting the number of occurrences of the associated words in the associated word set under each classification category in the text to be corrected.

[0085] When analyzing the possibility of the category to which the text to be corrected actually belongs from the perspective of text semantics, for each classification category, each word in the text to be corrected will first be checked to determine whether the word is a related word in the related word set under the classification category, thereby analyzing the number of times each related word in the related word set under the classification category appears in the text to be corrected, and the number of times the related words in the related word set under the classification category appear in the text to be corrected can be counted. The above process is performed for each classification category, and the number of times the related words in the related word set under each classification category appear in the text to be corrected can be counted, so as to subsequently analyze the possibility of the text to be corrected belonging to each classification category.

[0086] S340, calculating the second classification support rate of the text to be corrected in each classification category according to the number of occurrences in each classification category and the total number of words in the text to be corrected.

[0087] Optionally, after obtaining the number of occurrences of the associated words in the associated word set under each classification category in the text to be corrected, the proportion of the number of occurrences under each classification category to the total number of words in the text to be corrected is calculated, and the proportion can represent the possibility that the text to be corrected belongs to the classification category, that is, the second classification support rate of the text to be corrected under the classification category. For each classification category, the same steps as above are performed to calculate the second classification support rate of the text to be corrected under each classification category.

[0088] For example, assuming that the associated vocabulary set under a certain classification category contains three associated words A, B, and C, and the text to be corrected is ACDHEBASCBKDA, then it can be determined that the total number of words in the text to be corrected is 13, and the three associated words A, B, and C contained in the associated vocabulary set under the classification category appear in the text to be corrected in the order of: A, C, B, A, C, B, A, so the number of appearances of the associated words in the associated vocabulary set under the classification category in the text to be corrected is 7, and therefore the second classification support rate of the text to be corrected under this classification category is 7 / 13.

[0089] In order to ensure the comprehensiveness of the associated vocabulary set under each classification category, the present application will analyze the replaceable vocabulary of the category name of each classification category before predicting the second classification support rate of the text to be corrected under each classification category to construct the associated vocabulary set under the classification category. In other words, by analyzing various words in a large amount of text that have similar meanings to the category name of each classification category, which can be used as replaceable vocabulary for the category name of the classification category, and then combining various replaceable vocabulary, the associated vocabulary set under the classification category can be constructed.

[0090] Exemplarily, when analyzing the replaceable words of the category name of each classification category, for each classification category, the target text under the classification category is found from the constructed text training library, and the target text contains the category name of the classification category; the category name of the classification category is hidden in each target text, and each hidden target text is input into the constructed semantic replacement model to output the replaceable words of the category name of the classification category.

[0091] Specifically, considering that the same classification category may use different vocabulary representations in texts with different semantics, in order to accurately analyze the different vocabulary representations of texts with different semantics under the same classification category, a semantic replacement model will be pre-constructed, which can accurately analyze the semantics of each input text, and then output various words suitable for filling in the blank positions of the text according to the semantic information of each text. Among them, the semantic replacement model in the present application can be a bidirectional encoder representation (Bidirectional Encoder Representation from Transformers, BERT) model, which uses the masked language model (Masked Language Model, MLM) set in BERT to understand the semantics of the blank positions in each input text and predict the various words that can be replaced.

[0092] At this time, in order to conduct a comprehensive replacement analysis on the category name of each classification category, for each classification category, it is first necessary to find each target text containing the category name of the classification category from the constructed text training library. Then, Figure 5As shown, in each target text found under each classification category, the category name of the classification category is hidden, so that the target text appears vacant at the position of the category name of the classification category. Furthermore, for each classification category, each target text with the hidden category name under the classification category can be input into the constructed semantic replacement model respectively, and the semantic replacement model is used to perform semantic analysis on the vacant position that appears after the category name of the classification category is hidden in each target text, so that for each target text, a variety of various words suitable for filling in the vacant position in the target text can be output, and the output various words are used as replaceable words for the category name in the target text. In other words, for each target text under a certain classification category, the semantic replacement model will output multiple replaceable words for the category name of the classification category, so for each classification category, for each target text under the classification category, the TOP50 algorithm will be used to obtain the replaceable words in the target text that are most similar to the category name of the classification category. Then, each replaceable word obtained by the TOP50 algorithm for each target text under the classification category is merged, and according to the frequency of each replaceable word, combined stop words, etc., the TOP100 algorithm is used to continue to obtain a corresponding number of replaceable words from the merged replaceable words to construct a related word set under the classification category. The above process is performed for each classification category to construct a related word set under each classification category.

[0093] S350: Correct the marked classification labels of the text to be corrected by using the first classification support rate and the second classification support rate of the text to be corrected in each classification category.

[0094] The technical solution provided in the embodiment of the present application calculates the first classification support rate of the text to be corrected in each classification category through the classification results of the text to be corrected in each text classifier, and predicts the second classification support rate of the text to be corrected in each classification category through the association frequency of the associated vocabulary set under each classification category in the text to be corrected, thereby analyzing the actual classification situation of the text to be corrected from the two perspectives of the text classifier and the associated vocabulary set, thereby correcting the marked classification labels of the text to be corrected to ensure the accuracy of the text classification labels. Subsequently, such corrected text can be used to train the corresponding text classification model, thereby improving the classification accuracy of the trained text classification model.

[0095] Figure 6 A functional block diagram of a text classification label correction device shown in an embodiment of the present application.

[0096] like Figure 6 As shown, the apparatus 600 may include:

[0097] A text classification calculation module 610 is used to calculate the first classification support rate of the text to be corrected in each set classification category based on the classification results of the text to be corrected in each trained text classifier;

[0098] A text classification prediction module 620, configured to predict a second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected;

[0099] The classification label correction module 630 is used to correct the classification labels marked on the text to be corrected by using the first classification support rate and the second classification support rate of the text to be corrected in each classification category.

[0100] Furthermore, the text classification calculation module 610 can be specifically used for:

[0101] Inputting the text to be corrected into each trained text classifier respectively to obtain a classification result matrix of the text to be corrected;

[0102] Based on the occurrence frequency of the category name of each classification category in the classification result matrix, the first classification support rate of the text to be corrected under the classification category is calculated.

[0103] Furthermore, the text classification label correction device 600 may further include:

[0104] The text extraction module is used to extract the corresponding test text sets in batches from the constructed text training library;

[0105] The classifier training module is used to train the corresponding text classifier using the test text sets extracted from each batch to obtain the trained text classifiers.

[0106] Furthermore, the test text sets extracted in different batches are different.

[0107] Furthermore, the text classification prediction module 620 can be specifically used for:

[0108] Counting the number of occurrences of the associated words in the associated word set under each classification category in the text to be corrected;

[0109] The second classification support rate of the text to be corrected in each classification category is calculated according to the number of occurrences in each classification category and the total number of words in the text to be corrected.

[0110] Furthermore, the text classification label correction device 600 may further include:

[0111] The vocabulary set building module is used to analyze the replaceable words of the category name of each classification category to build a related vocabulary set under the classification category.

[0112] Furthermore, the vocabulary building module can be specifically used for:

[0113] For each classification category, a target text under the classification category is searched from the constructed text training library, wherein the target text contains the category name of the classification category;

[0114] The category name of the classification category is hidden in each of the target texts, and each of the hidden target texts is input into the constructed semantic replacement model to output replaceable words for the category name of the classification category.

[0115] Furthermore, the semantic replacement model is a transformer-based bidirectional encoding representation BERT model.

[0116] Furthermore, the classification label correction module 630 may be specifically used for:

[0117] respectively determining a first category to which the to-be-corrected text has a maximum support rate within the first classification, and a second category to which the to-be-corrected text has a maximum support rate within the second classification;

[0118] If the first category and the second category are the same, the category label annotated on the text to be corrected is corrected to the category name of the same category.

[0119] Furthermore, the number of the trained text classifiers is greater than or equal to the number of set classification categories, and the text to be corrected is any text in the constructed text training library.

[0120] In an embodiment of the present application, a plurality of text classifiers are pre-trained, and a corresponding associated vocabulary set is constructed in advance for each set classification category, so that when the classification label of the text to be corrected is corrected, the first classification support rate of the text to be corrected in each classification category is calculated through the classification result of the text to be corrected in each text classifier, and the second classification support rate of the text to be corrected in each classification category is predicted through the association frequency of the associated vocabulary set under each classification category in the text to be corrected, so as to analyze the actual classification situation of the text to be corrected from the two perspectives of the text classifier and the associated vocabulary set, so as to correct the marked classification label of the text to be corrected to ensure the accuracy of the text classification label, and subsequently such corrected text can be used to train the corresponding text classification model, thereby improving the classification accuracy of the trained text classification model.

[0121] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 6 The device 600 shown can execute any method embodiment provided in the present application, and the aforementioned and other operations and / or functions of each module in the device 600 are respectively for implementing the corresponding processes in each method of the embodiment of the present application, and for the sake of brevity, they will not be repeated here.

[0122] The above describes the device 600 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0123] Figure 7 It is a schematic block diagram of an electronic device 700 provided in an embodiment of the present application.

[0124] like Figure 7 As shown, the electronic device 700 may include:

[0125] The memory 710 and the processor 720, the memory 710 is used to store the computer program and transmit the program code to the processor 720. In other words, the processor 720 can call and run the computer program from the memory 710 to implement the text classification label correction method in the embodiment of the present application.

[0126] For example, the processor 720 may be configured to execute the above method embodiments according to instructions in the computer program.

[0127] In some embodiments of the present application, the processor 720 may include but is not limited to:

[0128] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0129] In some embodiments of the present application, the memory 710 includes but is not limited to:

[0130] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0131] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 710 and executed by the processor 720 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0132] like Figure 7 As shown, the electronic device may also include:

[0133] The transceiver 730 may be connected to the processor 720 or the memory 710 .

[0134] The processor 720 may control the transceiver 730 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 730 may include a transmitter and a receiver. The transceiver 730 may further include an antenna, and the number of antennas may be one or more.

[0135] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0136] The embodiment of the present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the embodiment of the present application also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0137] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0138] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0139] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0140] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0141] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for correcting text classification labels, characterized in that: include: Based on the classification results of the text to be corrected in each trained text classifier, calculating the first classification support rate of the text to be corrected in each set classification category; Predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected; Correcting the classification labels marked on the text to be corrected by using the first classification support rate and the second classification support rate of the text to be corrected in each classification category; The step of calculating the first classification support rate of the text to be corrected in each set classification category based on the classification result of the text to be corrected in each trained text classifier includes: Inputting the text to be corrected into each trained text classifier respectively to obtain a classification result matrix of the text to be corrected; Based on the frequency of occurrence of the category name of each classification category in the classification result matrix, calculating the first classification support rate of the text to be corrected under the classification category; The step of predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected includes: Counting the number of occurrences of the associated words in the associated word set under each classification category in the text to be corrected; The second classification support rate of the text to be corrected in each classification category is calculated according to the number of occurrences in each classification category and the total number of words in the text to be corrected.

2. The method according to claim 1, characterized in that Before calculating the first classification support rate of the text to be corrected in each set classification category based on the classification result of the text to be corrected in each trained text classifier, the method further includes: Extract the corresponding test text sets in batches from the constructed text training library; Using the test text sets extracted from each batch, respectively, to train the corresponding text classifiers to obtain the trained text classifiers; The test text sets extracted in different batches are different.

3. The method according to claim 1, characterized in that Before predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected, the method further includes: The replaceable words of the category name of each classification category are analyzed to construct a set of related words under the classification category.

4. The method according to claim 3, characterized in that: The analysis of replaceable words for the category name of each classification category includes: For each classification category, a target text under the classification category is searched from the constructed text training library, wherein the target text contains the category name of the classification category; The category name of the classification category is hidden in each of the target texts, and each of the hidden target texts is input into the constructed semantic replacement model to output replaceable words for the category name of the classification category.

5. The method according to claim 1, characterized in that The method of correcting the marked classification labels of the text to be corrected by using the first classification support rate and the second classification support rate of the text to be corrected in each classification category includes: respectively determining a first category to which the to-be-corrected text has a maximum support rate within the first classification, and a second category to which the to-be-corrected text has a maximum support rate within the second classification; If the first category and the second category are the same, the classification label annotated on the text to be corrected is corrected to the category name of the same category represented by the first category and the second category.

6. The method according to any one of claims 1 to 5, characterized in that: The number of the trained text classifiers is greater than or equal to the number of the set classification categories, and the text to be corrected is any text in the constructed text training library.

7. A text classification label correction device, characterized in that: include: A text classification calculation module, used to calculate the first classification support rate of the text to be corrected in each set classification category based on the classification result of the text to be corrected in each trained text classifier; A text classification prediction module, used for predicting the second classification support rate of the text to be corrected in each classification category based on the association frequency of the associated vocabulary set constructed in each classification category in the text to be corrected; A classification label correction module, used to correct the classification labels marked on the text to be corrected by using the first classification support rate and the second classification support rate of the text to be corrected in each classification category; Wherein, the text classification calculation module is specifically used for: Inputting the text to be corrected into each trained text classifier respectively to obtain a classification result matrix of the text to be corrected; Based on the frequency of occurrence of the category name of each classification category in the classification result matrix, calculating the first classification support rate of the text to be corrected under the classification category; The text classification prediction module is specifically used for: Counting the number of occurrences of the associated words in the associated word set under each classification category in the text to be corrected; The second classification support rate of the text to be corrected in each classification category is calculated according to the number of occurrences in each classification category and the total number of words in the text to be corrected.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the text classification label correction method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the text classification label correction method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for correcting a text classification label as described in any one of claims 1 to 6 is implemented.

Images