Text classification method, apparatus and electronic device

Abstract

The application discloses a text classification method, device and electronic equipment. The method comprises the following steps: obtaining a text to be classified, wherein the text to be classified carries a primary label and a secondary label; performing word segmentation on the text to be classified to obtain a plurality of first words corresponding to the text to be classified; determining a main component word of the text to be classified from the text to be classified according to the grammatical relationship between the first words in the text to be classified; obtaining a first semantic vector of the main component word, wherein the first semantic vector indicates the semantics of the main component word; more reliable information can be extracted for the classification of the text to be classified by obtaining the main component word in the text to be classified; the meaning expressed by the text to be classified can be known in a more fine-grained manner by obtaining the semantic vector of the main component word in the text to be classified; and finally, the first class label to which the text to be classified belongs is determined from the primary label and the secondary label based on the first semantic vector, thereby improving the classification accuracy of the text to be classified.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method, apparatus and electronic device for text classification. Background Technology

[0002] Text classification refers to classifying texts based on their similarity. Self-supervised methods are commonly used for text classification because they do not require a training process or pre-labeling of documents, thus offering flexibility and high automation. This has made it an important tool for effective text classification.

[0003] In related technologies, when classifying text, only the first-level labels are used as the basis for text classification. Using only the first-level labels as the basis for text classification can easily lead to low accuracy in text classification. Therefore, how to classify text more accurately is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for text classification to improve the accuracy of text classification.

[0005] In a first aspect, this application provides a method for text classification, comprising: obtaining a text to be classified, the text carrying a primary label and a secondary label, wherein the secondary label is a sub-label of the primary label; segmenting the text to be classified to obtain a plurality of first words corresponding to the text to be classified; determining the principal component words of the text to be classified from the text to be classified based on the grammatical relationship between the first words in the text to be classified, wherein the principal component words are at least one of the first words; obtaining a first semantic vector of the principal component words, the first semantic vector indicating the semantics of the principal component words; and determining a first category label to which the text sentence belongs based on the first semantic vector from the primary label and the secondary label.

[0006] Secondly, this application provides a text classification apparatus, comprising: an acquisition module for acquiring a text to be classified, the text carrying a primary label and a secondary label, wherein the secondary label is a sub-label of the primary label; a word segmentation module for segmenting the text to be classified to obtain a plurality of first words corresponding to the text to be classified; a determination module for determining the principal component words of the text to be classified from the text to be classified based on the grammatical relationship between the first words in the text to be classified, wherein the principal component words are at least one of the first words; the acquisition module is further configured to acquire a first semantic element vector of the principal component words, the first semantic element vector indicating the semantics of the principal component words; the determination module is further configured to determine a first category label to which the text to be classified belongs based on the first semantic element vector from the primary label and the secondary label.

[0007] Thirdly, this application provides an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method as described in the first aspect.

[0008] Fourthly, this application provides a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method as described in the first aspect.

[0009] As can be seen, by acquiring the text to be classified, which carries primary and secondary labels (the secondary labels being sub-labels of the primary labels), word segmentation is performed on the text to be classified, resulting in multiple primary words corresponding to the text to be classified. Then, based on the grammatical relationships between the primary words in the text to be classified, the principal component words of the text to be classified are determined. The principal component word is at least one of the primary words. The first semantic vector of the principal component word is then obtained. The first semantic vector indicates the semantics of the principal component word. In other words, by acquiring the principal component words in the text to be classified, we can focus on the main clause of the sentence and extract more reliable information for the classification of the text to be classified. By acquiring the semantic vector of the principal component words in the text to be classified, we can understand the meaning expressed by the text to be classified in a more granular way. Finally, based on the first semantic vector, the first category label to which the text to be classified belongs is determined from the primary and secondary labels. That is, the semantics indicated by the semantic vector of the principal component words in the text to be classified are used as the classification basis to select the category label that is more suitable for the text to be classified from the primary and secondary labels, thus improving the classification accuracy of the text to be classified. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0011] Figure 1 A flowchart illustrating a text classification method provided in an embodiment of this application;

[0012] Figure 2 A schematic diagram of the structure of a text classification device provided in an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of an electronic device provided as an embodiment of the present specification. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0015] The terms "first," "second," etc., used in this specification and claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in this specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0016] As mentioned earlier, text classification only uses the first-level labels as the basis for text classification. However, the first-level labels can be further subdivided into second-level labels to achieve a more granular classification of the text. Currently, text classification only uses the first-level labels as the basis for text classification, resulting in low accuracy. Therefore, how to classify text more accurately is a technical problem that needs to be solved by those skilled in the art.

[0017] To improve the accuracy of text classification, this application aims to provide a text classification scheme, which includes: obtaining a text to be classified, the text carrying a primary label and a secondary label, wherein the secondary label is a sub-label of the primary label; segmenting the text to be classified to obtain a plurality of first words corresponding to the text to be classified; determining the principal component words of the text to be classified from the text to be classified based on the grammatical relationship between the first words in the text to be classified, wherein the principal component words are at least one of the first words; obtaining the first semantic element vector of the principal component words, the first semantic element vector indicating the semantics of the principal component words; and determining the first category label to which the text sentence belongs based on the first semantic element vector from the primary label and the secondary label.

[0018] The technical solution disclosed in this application involves obtaining a text to be classified, which carries primary and secondary labels, with the secondary labels being sub-labels of the primary labels. The text is then segmented to obtain multiple first words corresponding to the text. Based on the grammatical relationships between these first words, the main constituent words of the text are determined. Each main constituent word is at least one of the first words. The first semantic vector of each main constituent word is then obtained, indicating its semantic meaning. In other words, by obtaining the main constituent words, the core of the sentence can be identified, providing more reliable information for classification. Furthermore, obtaining the semantic vectors of the main constituent words allows for a more granular understanding of the meaning expressed by the text. Finally, based on the first semantic vectors, the first category label of the text is determined from the primary and secondary labels. This means that the semantic meaning indicated by the semantic vectors of the main constituent words is used as the classification criterion to select a more suitable category label from the primary and secondary labels, thus improving the classification accuracy of the text.

[0019] It should be understood that the text classification methods provided in the embodiments of this application can all be executed by an electronic device or by software installed in an electronic device, specifically by a terminal device or a server device. The text processing methods can be executed by the same electronic device or by different electronic devices.

[0020] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0021] Please refer to Figure 1 This is a flowchart illustrating a text classification method provided in one embodiment of this specification, applied to an electronic device. The method may include:

[0022] Step S101: Obtain the text to be classified.

[0023] The text to be classified carries a first-level tag and a second-level tag, where the second-level tag is a sub-tag of the first-level tag.

[0024] Specifically, the text to be classified is labeled data containing text and corresponding tags. The text to be classified can consist of at least one sentence, and each sentence consists of multiple words. The tags carried by the text to be classified are multi-level progressive tags, namely, primary tags and secondary tags. Primary tags can be further subdivided into multiple secondary tags. For example, if the text to be classified is "I want to repay early," its primary tag could be "repayment on installment," and its secondary tag could be "repayment early," where "repayment early" is a sub-tag within "repayment on installment." It is worth noting that a primary tag can have multiple secondary tags, and secondary tags can be further divided into multiple sub-tags. These tags can all be classified under the primary secondary tags. That is, in this embodiment, the secondary tags include the secondary tag itself and its sub-tags, etc., but this embodiment does not limit this.

[0025] Step S103: Segment the text to be classified into words to obtain multiple first words corresponding to the text to be classified.

[0026] Specifically, the text to be classified can consist of at least one sentence, each sentence consists of multiple words, each word has a different part of speech, and words with different parts of speech serve different sentence components in the sentence. The text to be classified can be segmented into multiple first words according to the part of speech and sentence component of the words. Each first word has its own part of speech and the sentence component it serves.

[0027] Step S105: Determine the main constituent words of the text to be classified from the text to be classified based on the grammatical relationship between each first word in the text to be classified.

[0028] Among them, the main component word is at least one of the first words.

[0029] Specifically, each first word is a component of a sentence in the text to be classified. The sentences in the text to be classified include first words of different parts of speech, such as nouns, pronouns, verbs, adjectives, quantifiers, and adverbs. Each first word of a different part of speech serves as a different sentence component in the text. The grammatical relationships between the first words in the text refer to the relationships between different sentence components, such as subject-predicate, verb-object, coordinate, and modifying relationships. Subject-predicate relationships refer to the relationship between the subject and predicate; verb-object relationships refer to the relationship between the verb and object; coordinate relationships refer to the parallel arrangement of content words with the same part of speech; and modifying relationships refer to the preceding word modifying or restricting the following word. Furthermore, among the sentence components that make up the sentences in the text to be classified, the first word corresponding to at least one sentence component that can express the important meaning of the text to be classified can be used as the principal component word. For example, the first words corresponding to the subject, predicate, and headword components in the sentence of the text to be classified can be used as principal component words.

[0030] In one possible implementation, determining the principal components of the text to be classified based on the grammatical relationships between the first words in the text to be classified includes: performing syntactic analysis on the grammatical relationships between the first words in the text to be classified to obtain the grammatical structure information of the text to be classified; inputting the text to be classified into a classic automatic text summarization model for summary extraction to obtain a summary of the text to be classified, wherein the grammatical structure information indicates the part of speech of each first word; searching for modifiers in a modifier template library from the summary based on the grammatical structure information to obtain the modifiers of the text to be classified, wherein the modifier template library is pre-constructed; deleting the modifiers of the text to be classified to obtain text tags; and inputting the text tags and the text to be classified into a pre-trained language model to obtain the principal components of the text to be classified, wherein the pre-trained language model is used to perform principal component analysis on the text to be classified using the text tags to obtain the principal components of the text to be classified.

[0031] Specifically, syntactic analysis refers to analyzing the grammatical functions of words in sentences of a text to be classified, that is, analyzing the sentence components that each word in the text acts as. The grammatical structure information of the text to be classified refers to the part of speech of each first word that acts as a different sentence component in the sentences of the text to be classified. The classic automatic text summarization model (Textsum) is used to extract summaries from the text to be classified, resulting in a summary of the text. This summary reflects the outline of the text to be classified. The sentence components that the first words of different parts of speech in the summary may include modifiers, such as adverbs and attributives. These modifiers correspond to words of the corresponding parts of speech and do not affect the main expression of the text to be classified. The modifier template library refers to the sentence components that function as modifiers and the parts of speech of the words corresponding to these sentence components. This library can be pre-constructed by relevant personnel for subsequent use. Therefore, the modifiers in the summary can be determined according to the analyzed grammatical structure information, and then the modifiers in the summary can be searched from the modifier template library. If the modifier in the summary appears in the modifier template library, the modifier in the summary is deleted, and the words corresponding to the remaining sentence components in the summary are used as the initial principal components.

[0032] The modifier template library may not fully cover all modifiers appearing in the abstract. Furthermore, the principal component words identified through the modifier template library may still include modifiers. Therefore, to further supplement the principal component words in the text to be classified and improve their accuracy, further extraction of principal component words can be performed. Specifically, text labels and the text to be classified are input into a pre-trained language model (Bart). The pre-trained language model (Bart) performs principal component analysis on the text to obtain the principal component words. Specifically, the pre-trained language model performs principal component analysis on the text to be classified by determining the sentence components corresponding to the text labels and their corresponding first words from the text to be classified according to the text labels. Then, it performs principal component analysis on the sentence components corresponding to the text labels, removing the modifiers. The first words corresponding to the remaining sentence components in the text to be classified are then used as the principal component words of the text to be classified. This method yields more accurate principal component words for the text to be classified, facilitating precise classification.

[0033] Furthermore, the pre-trained language model can be pre-trained, specifically by acquiring text samples; performing syntactic analysis on the grammatical relationships between the second words in the text samples to obtain the sample grammatical structure information, which indicates the part of speech of each second word; inputting the text samples into a classic automatic text summarization model for summary extraction to obtain a summary of the text samples; deleting modifiers in the summary based on the sample grammatical structure information to obtain sample labels, where the modifiers are at least one of the pre-constructed modifier templates; and inputting the sample labels and text samples into the pre-trained language model to be trained to obtain the pre-trained language model.

[0034] Specifically, a text sample refers to the text used to train a pre-trained language model. The text includes at least one sentence, and each sentence is segmented to obtain multiple second words corresponding to the text sample.

[0035] Syntactic analysis refers to the analysis of the grammatical functions of words in a text sample sentence, that is, the analysis of the sentence components that each word in the text sample functions as. The sample grammatical structure information of the text sample refers to the part of speech of each second word that functions as a different sentence component in the text sample sentence. The classic automatic text summarization model (Textsum) is used to extract a summary of the text sample. This summary reflects the outline of the text sample. The sentence components that the second words of different parts of speech in the summary may include modifiers, which can be adverbs or attributives. These modifiers correspond to words of the corresponding parts of speech and do not affect the main expression of the text sample. The modifier template library refers to the sentence components that function as modifiers and the parts of speech of the words corresponding to the sentence components. It can be pre-constructed by relevant personnel for subsequent use. Therefore, the modifiers in the summary can be determined according to the analyzed sample grammatical structure information, and then the modifiers in the summary can be searched from the modifier template library. If the modifier in the summary appears in the modifier template library, the modifier in the summary is deleted, and the words corresponding to the remaining sentence components in the summary are used as the initial principal components.

[0036] The modifier template library may not fully cover all modifiers appearing in the abstract. Furthermore, the principal component words identified through the template library may also include modifiers. Therefore, to further supplement the principal component words in the text samples and improve their accuracy, further extraction of principal component words can be performed. Specifically, text labels and text samples are input into a pre-trained language model (bart) for training. During training, the pre-trained language model (bart) uses the text labels to perform principal component analysis on the text samples, obtaining the principal component words and enabling the trained language model to perform principal component analysis.

[0037] Step S107: Obtain the first semantic vector of the main component word, which indicates the semantics of the main component word.

[0038] Specifically, the first semantic vector of the main component word can be obtained through a semantic vector table, or the semantic vector of the main component word can be obtained through semantic analysis. The semantic vector represents the semantics of the main component word; it refers to the representation of the semantic features of the main component in vector form. The semantic vector table can be determined as follows:

[0039] 1) First, load the two-dimensional table of semantic vectors for the general domain.

[0040] 2) A two-dimensional table of semantic vectors for domain-specific words, manually annotated, serving as the basic semantic elements (explicit semantic elements) of the words.

[0041] 3) Construct a task to compare synonyms and antonyms of proper words in a specific domain, and label the semantic distinguishing feature sequences of proper words in the specific domain.

[0042] 4) Construct a sentence comparison task within the domain (the first sentence is a legal sentence, the second sentence is an illegal sentence, and the two differ by only one word), label the unique distinguishing semantic feature sequence, and form a special semantic feature list for semantic feature classes that do not belong to the basic semantic feature list, thereby mining latent semantic features.

[0043] 5) Distribute all semantic features in the semantic feature list into syntactic semantic feature space, categorical semantic feature space, and connotative semantic feature space according to human understanding. The syntactic semantic feature space includes: part of speech, legal syntactic collocation slots, and illegal syntactic collocation slots; the categorical semantic feature space includes: biological and non-biological; and the connotative semantic feature space includes: time dimension, spatial dimension, color dimension, etc. Draw a high-dimensional semantic planet in multiple spaces so that the semantic distance between semantic feature vectors (generalized semantic distance, including syntactic distance) is truly reflected in the semantic planet.

[0044] 6) Training: The previously labeled sentence comparison data is used to train the model (based on the multi-classification task fine-tuning of the pre-trained language model) (X is a pair of sentences, the first sentence is a legal sentence and the second sentence is an illegal sentence, and the two differ by only one word), and y is the semantic distinguishing sequence of the two words that are the only difference.

[0045] 7) Put all the semantic class names and semantic vectors during the model training process into a two-dimensional table to form a semantic two-dimensional table, ensuring that one semantic element corresponds to one semantic vector, and construct a semantic mapping table.

[0046] 8) Following step 6), concatenate all the semantic feature sequences contained in the same word from left to right, and initialize each space in a fixed order according to the order of grammatical semantic feature space, categorical semantic feature space, and connotative semantic feature space. Concatenate all the semantic feature vectors of each word in all semantic feature dimensions (each semantic feature vector is represented by one dimension; if there is no semantic feature, the value of the semantic feature dimension is 0; if the opposite is true, the semantic feature value is negative).

[0047] 9) Generate the summation relationship between words and semantic vectors, establish the summation equation between words and semantic vectors, and establish a mapping relationship table between words and semantic vectors, i.e., the semantic vector table.

[0048] Step S109: Determine the first category label to which the text to be classified belongs from the first-level label and the second-level label based on the first semantic vector.

[0049] Specifically, after determining the primary semantic element vector, the vector distance between the primary semantic element vector and the primary and secondary labels can be calculated. This vector distance determines the primary category label to which the text to be classified belongs. The smaller the distance between the primary semantic element vector and a particular label, the more likely the text belongs to that label. Specifically, this can be calculated between the primary semantic element vector and the word vectors corresponding to the primary and secondary labels; or between the primary semantic element vector and the final vector composed of the word vectors and semantic element vectors corresponding to the primary or secondary labels; or between the word vectors or semantic element vectors of the main component words and the final vector composed of the word vectors and semantic element vectors corresponding to the primary or secondary labels.

[0050] In one possible implementation, determining the first category label of the text to be classified based on the first semantic vector from the first-level label and the second-level label includes: obtaining the first word vector of the main component words; obtaining the second word vector and the second semantic vector of the first-level label, as well as the third word vector and the third semantic vector of the second-level label, where the second semantic vector indicates the semantics of the first-level label and the third semantic vector indicates the semantics of the second-level label; determining the first vector of the first-level label based on the second word vector and the second semantic vector, and determining the second vector of the second-level label based on the third word vector and the third semantic vector; and classifying the text to be classified based on the first word vector, the first semantic vector, the first vector, and the second vector to obtain the first category label.

[0051] Specifically, the word vectors of principal component words, first-level labels, and second-level labels can be obtained through a word vector table, which can be obtained through a deep learning model. The semantic element vectors of first-level labels and second-level labels can be obtained through the semantic element vector table generated in the above embodiments or through semantic element analysis. The semantic element vectors represent the semantics of the principal component words, meaning that the semantic elements of the principal component are represented in vector form. Specifically, after obtaining the second word vector and the second semantic element vector of the first-level label, the second word vector and the second semantic element vector are added together to obtain the first vector. Similarly, the third word vector and the third semantic element vector of the second-level label are added together to obtain the second vector. Thus, when determining the category label of a text to be classified, the category label can be determined according to the semantic feature vectors of the principal components and the semantic feature vectors of the first-level or second-level labels in the text to be classified. Since the semantic feature vectors of the principal components and the semantic feature vectors of the first-level or second-level labels can both represent their respective semantics, and the semantics indicated by the semantic feature vectors of the principal components and the semantics indicated by the semantic feature vectors of the first-level or second-level labels in the text to be classified are used as the classification basis, the classification accuracy of the text to be classified can be further improved.

[0052] It is worth noting that there are other ways to obtain the word vectors and semantic vectors mentioned above, and this application embodiment does not limit them.

[0053] In one possible implementation, classifying the text to be classified based on the first word vector, the first semantic element vector, the first vector, and the second vector to obtain the first category label includes: calculating the first distance between the first word vector and the first vector respectively; if the first distance is less than a first threshold, calculating the second distance between the first word vector and the second vector, and the third distance between the first semantic element vector and the second vector respectively; if the second distance is less than the second threshold and the third distance is less than the third threshold, determining the first category label to which the text to be classified belongs as a second-level label; if the second distance is not less than the second threshold or the third distance is not less than the third threshold, determining the first category label to which the text to be classified belongs as a first-level label.

[0054] Specifically, the first threshold, second threshold, and third threshold can be determined according to the actual situation. This application embodiment does not limit the specific values ​​of the first to second thresholds. When calculating the distance between the vectors, Euclidean distance, cosine distance, and Manhattan distance can be calculated, and this application embodiment does not limit these calculations. More specifically, when the first distance is less than the first threshold, it indicates that the text to be classified belongs to a first-level label. A fourth distance can be further calculated between the first word vector and the first semantic vector and the second vector to determine whether the text to be classified belongs to a second-level label. If the calculated second distance is less than the second threshold and the third distance is less than the third threshold, it indicates that the text to be classified belongs to a second-level label. If either the second or third distance is not less than the corresponding threshold, it indicates that the text to be classified does not belong to a second-level label but to a first-level label. When the first distance is not less than the first threshold, it indicates that the text to be classified does not belong to a first-level label.

[0055] Thus, when determining whether a text to be classified belongs to a secondary label, the vector distance at the semantic feature level is used. This is because the semantic granularity of secondary labels is smaller, and using the distance at this smaller semantic granularity of semantic features can effectively distinguish the commonalities and differences between secondary and primary labels, further improving the accuracy of classifying the text in primary and secondary labels.

[0056] The technical solution disclosed in this application involves obtaining a text to be classified, which carries primary and secondary labels, with the secondary labels being sub-labels of the primary labels. The text is then segmented to obtain multiple first words corresponding to the text. Based on the grammatical relationships between these first words, the main constituent words of the text are determined. Each main constituent word is at least one of the first words. The first semantic vector of each main constituent word is then obtained, indicating its semantic meaning. In other words, by obtaining the main constituent words, the core of the sentence can be identified, providing more reliable information for classification. Furthermore, obtaining the semantic vectors of the main constituent words allows for a more granular understanding of the meaning expressed by the text. Finally, based on the first semantic vectors, the first category label of the text is determined from the primary and secondary labels. This means that the semantic meaning indicated by the semantic vectors of the main constituent words is used as the classification criterion to select a more suitable category label from the primary and secondary labels, thus improving the classification accuracy of the text.

[0057] To further improve the classification accuracy of the text to be classified, the classification can be performed by combining the context and the main components of the text. In one possible implementation, after determining the first category label of the text to be classified based on the first semantic vector from the first-level label and the second-level label, the method further includes: obtaining the sentence identifier and position information of each first word in the text to be classified; obtaining the first initial word vector of each first word, and obtaining the fourth semantic vector of each first word respectively; superimposing the first initial word vector, the fourth semantic vector, the sentence identifier and the position information of each first word to obtain the fourth word vector corresponding to each first word; classifying the text to be classified according to the fourth word vector, the first vector and the second vector of each first word to obtain the second category label of the text to be classified; calculating the weighted sum of the first category label and the second category label to obtain the target category of the text to be classified.

[0058] Specifically, for the first word constituting the text to be classified, it carries the sentence identifier (segmentation embedding) of the text it belongs to and the position information (position embedding) of the first word within that sentence. The sentence identifier (segmentation embedding) indicates which sentence the first word is in, and the position information (position embedding) indicates the specific position of the first word within that sentence. The initial word vector (word_embedding) of the first word can be obtained from a word vector table, which can be obtained through a deep learning model. The semantic vector (semely_word_embedding) of the first word can be obtained from the semantic vector table generated in the above embodiment. The fourth word vector of the first word vector can be obtained by adding the initial word vector (word_embedding), the fourth semantic vector (semely_word_embedding), the sentence identifier (segmentation embedding), and the position information (position embedding). Then, the vector distances between the fourth word vector and the first vector, and between the fourth word vector and the second vector, are calculated to classify the text to be classified, obtaining the second category label to which the text belongs.

[0059] Further, the specific steps for weighting the first category label and the second category label are as follows: Map the first category label to a first numerical value and the second category label to a second numerical value using a mapping dictionary; obtain the first weight assigned to the first category label and the second weight assigned to the second category label; calculate the sum of the product of the first numerical value and the first weight and the product of the second numerical value and the second weight to obtain the weighted sum. If the weighted sum is greater than a sixth threshold, the final category of the text to be classified is a first-level label; if the weighted sum is not greater than the sixth threshold, the final category of the text to be classified is a second-level label. The first weight can be greater than the second weight, and the sixth threshold can be determined according to the actual situation; this embodiment does not limit this. Furthermore, the numerical value mapped to the first-level label is greater than the numerical value mapped to the second-level label. When the first category label and the second category label are the same (i.e., both belong to the first-level label or the second-level label), the first numerical value and the second numerical value are the same. Thus, by combining the semantic features of the entire text to be classified with the semantic features of its main constituent words, classification can be performed. By obtaining the main constituent words in the text to be classified, we can focus on the sentence structure and extract more reliable information for the classification of the text to be classified. By obtaining the semantic feature vectors of the entire text to be classified and the semantic feature vectors of the main constituent words, we can focus on the semantic information expressed by the entire text and its main structure, thereby understanding the meaning expressed by the text to be classified in a more granular and accurate way. Therefore, using the semantic features indicated by the semantic feature vectors of the main constituent words and the semantic features indicated by the semantic feature vectors of the entire text to be classified as the classification basis, we select the category label that is more suitable for the text to be classified from the primary and secondary labels, making the classification of the text to be classified more accurate.

[0060] In one possible implementation, classifying the text to be classified based on the fourth word vector, the first vector, and the second vector to obtain the second category label to which the text to be classified belongs includes: calculating the fourth distance between the fourth word vector and the first vector; if the fourth distance is less than the fourth threshold, calculating the fifth distance between the fourth word vector and the second vector; if the fifth distance is less than the fifth threshold, determining the second category label to which the text to be classified belongs as a secondary label; if the fifth distance is not less than the fifth threshold, determining the second category label to which the text to be classified belongs as a primary label.

[0061] Specifically, when calculating the distance between the aforementioned vectors, Euclidean distance, cosine distance, and Manhattan distance can be calculated, and this embodiment does not limit the calculation. More specifically, when the fourth distance is less than the fourth threshold, it indicates that the text to be classified belongs to a first-level label. A fifth distance can be further calculated between the fourth word vector and the second vector to determine whether the text to be classified belongs to a second-level label. If the calculated fifth distance is less than the fifth threshold, it indicates that the text to be classified belongs to a second-level label; if the fifth distance is not less than the fifth threshold, it indicates that the text to be classified does not belong to a second-level label but belongs to a first-level label. When the fourth distance is not less than the fourth threshold, it indicates that the text to be classified does not belong to a first-level label. It is worth noting that the fourth and fifth thresholds can be determined according to the actual situation, and this embodiment does not limit the specific values ​​of the fourth and fifth thresholds. Thus, when determining whether the text to be classified belongs to a second-level label, the vector distance at the semantic feature level is used. This is because the semantic granularity of second-level labels is small, and using distance at this smaller semantic granularity can effectively distinguish the commonalities and differences between second-level and first-level labels, further improving the accuracy of classifying the text between first-level and second-level labels.

[0062] The technical solution provided in this application embodiment will be further described in detail below with reference to a specific application scenario. The specific application scenario is chatbot silence detection. Silence detection classifies silent data in chatbot conversation data. Existing silence detection models predict the primary category to which the silent data belongs. This application embodiment performs secondary classification based on the primary classification, manual data annotation, text, and text context information, achieving a more accurate classification of chatbot silence data, thereby providing users with accurate business recommendations and improving user experience. In this specific application scenario, the text to be classified is the dialogue text between the chatbot and the user. This dialogue text is considered silent data and includes the business information inquired by the user. Through the technical solution provided in this application embodiment, the business category to which the business information in the dialogue text belongs is determined. The first category label is the business category label to which the business information belongs, thereby recommending target businesses related to the business category label to the user. The target business can be all businesses related to the business inquired by the user, or it can be a specific business inquired by the user. This application embodiment does not limit this.

[0063] In this application scenario, the silent data between the chatbot and the user is acquired. This silent data can be a single text, the original tags corresponding to the single text, or a summary of manual analysis. For example, 1) the silent data includes the following:

[0064] a. Text content: 'Hello sir, I will now process your refunds one contract at a time. Thank you for your patience.'

[0065] b. Contextual Sentence: 'I know you just saw that, sir, please wait a moment. There are quite a few records here, let me check them for you, just a moment.' msxf 'Okay. Okay, okay.' msxf 'Hello, sir, I'll refund you one contract at a time now. Thank you for your patience.' msxf 'Uh, yes, so can I transfer the money in now?' msxf 'Yes.'

[0066] c. Original tag: Other silent categories.

[0067] d. Manual analysis summary: Similar to the previous "2", this occurred during the same silent waiting phase. The agent informed the customer that there was a lot of data and asked them to wait a little longer.

[0068] 2) Input the manually analyzed minutes into the summary generation model (e.g., PreSumm or Pegasus), and retrieve the top_n summary results. For example: Summary 1: Due to the large amount of data, please ask the customer to wait; Summary 2: The agent informed the customer that there is a large amount of data and asked them to wait.

[0069] a. Prepare labeled data: Combine individual texts with contextual information and manually analyzed summaries to form the original text. Please manually label the text as silent categories.

[0070] Concatenation method: Use the [sep] separator to concatenate the single text with contextual information with the manually analyzed summary text. Specific examples are shown in the labeled data below.

[0071] b. Input the labeled data into the PreSumm model according to the following format, and train the model based on the pre-trained model.

[0072] The data annotation format can be as follows: "text" or "abstract".

[0073] 'I know you just saw that, sir, please wait a moment. There are quite a few records here, let me check them for you, just a moment.' msxf 'Okay.' 'Okay, okay.' msxf 'Hello, sir, I'll refund you one contract at a time now. Thank you for your patience.' msxf 'Yes, that's me. Can I transfer the money now?' msxf 'Okay.' '[sep] and the previous "2" are in the same silent waiting phase, the agent informs the customer that there is a lot of data and asks them to wait a little longer; due to the large amount of data, the customer is asked to wait.'

[0074] 3) Combine the summary results with the original tags to generate pre-tags for a single text. For example: Due to the large amount of data, please wait for the customer. Other categories are silent. The combination method can be: directly concatenate the summary results and the original tags using an underscore "_" as a connector.

[0075] 4) Input a single text into the principal component analysis model to obtain the principal component words of the single text.

[0076] 5) Decompose the pre-label into business elements, and the operation repayment is less than the business operation (that is, the operation repayment is a sub-business of the business operation).

[0077] 6) Obtain the semantic feature vectors of the main component words, and determine the first category label of the text to be classified from the pre-labels based on the semantic feature vectors. The category label of the text to be classified is determined from the pre-labels according to the semantics expressed by the semantic feature vectors. For example, if the category label of the above single text is determined to be "Slow Repayment, Please Wait" or "Silent Operation," it is determined to be a lower-level label of "Slow Business Operation, Please Wait." Thus, the category label of the silent data determined through this application embodiment belongs to the repayment category label. Therefore, repayment-related services are recommended to the user based on the repayment category label.

[0078] In addition, with the above Figure 1 Corresponding to the text classification method shown, this application also provides a text classification apparatus. Figure 2 This is a schematic diagram of the structure of a text classification device 200 provided in an embodiment of this application, including: an acquisition module 201, used to acquire text to be classified, the text to be classified carrying a first-level label and a second-level label, wherein the second-level label is a sub-label of the first-level label; a word segmentation module 202, used to segment the text to be classified to obtain multiple first words corresponding to the text to be classified; a determination module 203, used to determine the main component words of the text to be classified from the text to be classified according to the grammatical relationship between each first word in the text to be classified, wherein the main component words are at least one of the first words; the acquisition module 201 is also used to acquire the first semantic element vector of the main component words, the first semantic element vector indicating the semantics of the main component words; the determination module 203 is also used to determine the first category label to which the text to be classified belongs based on the first semantic element vector from the first-level label and the second-level label.

[0079] The text classification apparatus provided in this application obtains a text to be classified, which carries primary and secondary labels, with the secondary labels being sub-labels of the primary labels. The text is then segmented to obtain multiple first words corresponding to the text. Based on the grammatical relationships between these first words, the principal component words of the text are determined. Each principal component word is at least one of the first words. The first semantic vector of each principal component word is then obtained, indicating its semantic meaning. In other words, by obtaining the principal component words, the core of the sentence can be identified, providing more reliable information for classification. Furthermore, by obtaining the semantic vectors of the principal component words, the meaning expressed by the text can be understood more finely. Finally, based on the first semantic vectors, the first category label of the text is determined from the primary and secondary labels. That is, the semantic meaning indicated by the semantic vectors of the principal component words is used as the classification basis to select a more suitable category label from the primary and secondary labels, thus improving the classification accuracy of the text.

[0080] In one possible implementation, the determining module 203 is further configured to obtain the first word vector of the principal component words; obtain the second word vector and the second semantic element vector of the first-level label, as well as the third word vector and the third semantic element vector of the second-level label, wherein the second semantic element vector indicates the semantics of the first-level label and the third semantic element vector indicates the semantics of the second-level label; determine the first vector of the first-level label based on the second word vector and the second semantic element vector, and determine the second vector of the second-level label based on the third word vector and the third semantic element vector; classify the text to be classified based on the first word vector, the first semantic element vector, the first vector and the second vector to obtain the first category label.

[0081] In one possible implementation, the determining module 203 is further configured to calculate the first distance between the first word vector and the first vector respectively; if the first distance is less than the first threshold, calculate the second distance between the first word vector and the second vector, and the third distance between the first semantic vector and the second vector respectively; if the second distance is less than the second threshold and the third distance is less than the third threshold, determine the first category label to which the text to be classified belongs as a second-level label; if the second distance is not less than the second threshold or the third distance is not less than the third threshold, determine the first category label to which the text to be classified belongs as a first-level label.

[0082] In one possible implementation, the acquisition module 201 is further configured to acquire the sentence identifier and position information of the text to be classified where each first word is located; acquire the first initial word vector of each first word, and acquire the fourth semantic vector of each first word respectively; superimpose the first initial word vector, fourth semantic vector, sentence identifier and position information of each first word to obtain the fourth word vector corresponding to each first word; and further include: a classification module, configured to classify the text to be classified according to the fourth word vector, first vector and second vector of each first word to obtain the second category label to which the text to be classified belongs; calculate the weighted sum of the first category label and the second category label to obtain the target category of the text to be classified.

[0083] In one possible implementation, the classification module is further configured to calculate a fourth distance between the fourth word vector and the first vector; if the fourth distance is less than a fourth threshold, calculate a fifth distance between the fourth word vector and the second vector; if the fifth distance is less than a fifth threshold, determine the second category label to which the text to be classified belongs as a secondary label; if the fifth distance is not less than a fifth threshold, determine the second category label to which the text to be classified belongs as a primary label.

[0084] In one possible implementation, the determining module 203 is further configured to perform syntactic analysis on the grammatical relationships between the first words in the text to be classified, to obtain the grammatical structure information of the text to be classified, and to input the text to be classified into a classic automatic text summarization model for summary extraction, to obtain a summary of the text to be classified, wherein the grammatical structure information indicates the part of speech of each first word; to search for modifiers in the modifier template library from the summary based on the grammatical structure information, to obtain the modifiers of the text to be classified, wherein the modifier template library is pre-constructed; to delete the modifiers of the text to be classified, to obtain text tags; and to input the text tags and the text to be classified into a pre-trained language model, to obtain the principal component words of the text to be classified, wherein the pre-trained language model is used to perform principal component analysis on the text to be classified using the text tags, to obtain the principal component words of the text to be classified.

[0085] In one possible implementation, the acquisition module 201 is further configured to acquire text samples; perform syntactic analysis on the grammatical relationships between the second words in the text samples to obtain the sample grammatical structure information of the text samples, wherein the sample grammatical structure information indicates the part of speech of each second word; input the text samples into a classic automatic text summarization model for summary extraction to obtain a summary of the text samples; delete the modifiers in the summary according to the grammatical structure information to obtain sample labels, wherein the modifiers are at least one of the pre-constructed modifier templates; and input the sample labels and text samples into a pre-trained language model to be trained to obtain a pre-trained language model.

[0086] The text to be classified is the dialogue text between the chatbot and the user. The dialogue text includes the business information inquired by the user. The first category label is the business category label to which the business information belongs. It also includes a recommendation module, which is used to recommend target businesses related to the business category label to the user based on the business category label.

[0087] Obviously, the text classification apparatus disclosed in this application can serve as the execution subject of the text classification method shown in the above embodiments, and thus can realize the functions of the text classification method implemented in the above embodiments. Since the principle is the same, it will not be described again here.

[0088] Figure 3 This is a schematic diagram of the structure of an electronic device according to one embodiment of this specification. Please refer to it. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0089] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0090] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0091] The processor reads the corresponding computer program from non-volatile memory into memory and then runs it, forming a text classification device at the logical level. The processor executes the program stored in memory and specifically performs the text classification method mentioned in any of the above method embodiments.

[0092] The above is as described in this instruction manual. Figure 1The text classification apparatus disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0093] It should be understood that the electronic device in the embodiments of this application can implement a text classification device. Figure 1 The embodiments shown have the same function. Since the principle is the same, the embodiments of this application will not be described again here.

[0094] Of course, in addition to software implementation, the electronic device described in this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0095] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform the text classification method of any of the above embodiments.

[0096] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0097] In summary, the above are merely preferred embodiments of this specification and are not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0098] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0099] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0100] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0101] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method of text classification, characterized by, include: Obtain the text to be classified, which carries a primary tag and a secondary tag, wherein the secondary tag is a sub-tag of the primary tag; The text to be classified is segmented into words to obtain multiple first words corresponding to the text to be classified. The main constituent words of the text to be classified are determined from the grammatical relationships between the first words in the text to be classified, wherein the main constituent words are at least one of the first words; Obtain the first semantic element vector of the main component word, whereby the first semantic element vector indicates the semantics of the main component word; Based on the first semantic feature vector, a first category label to which the text to be classified belongs is determined from the first-level label and the second-level label; wherein, determining the first category label to which the text to be classified belongs from the first-level label and the second-level label based on the first semantic feature vector includes: Obtain the first word vector of the principal component words; Obtain the second word vector and the second semantic element vector of the first-level tag, as well as the third word vector and the third semantic element vector of the second-level tag, wherein the second semantic element vector indicates the semantics of the first-level tag, and the third semantic element vector indicates the semantics of the second-level tag; The first vector of the first-level label is determined based on the second word vector and the second semantic element vector, and the second vector of the second-level label is determined based on the third word vector and the third semantic element vector; The text to be classified is classified based on the first word vector, the first semantic element vector, the first vector, and the second vector to obtain the first category label.

2. The method of text classification according to claim 1, wherein, The process of classifying the text to be classified based on the first word vector, the first semantic element vector, the first vector, and the second vector to obtain the first category label includes: Calculate the first distance between the first word vector and the first vector respectively; If the first distance is less than the first threshold, calculate the second distance between the first word vector and the second vector, and the third distance between the first semantic element vector and the second vector. If the second distance is less than the second threshold and the third distance is less than the third threshold, the first category label to which the text to be classified belongs is determined to be the second-level label; If the second distance is not less than the second threshold or the third distance is not less than the third threshold, the first category label to which the text to be classified belongs is determined to be the first-level label.

3. The method of text classification according to claim 1, wherein, After determining the first category label to which the text to be classified belongs based on the first semantic vector from the first-level label and the second-level label, the method further includes: Obtain the sentence identifier and position information of each of the first words in the text to be classified; Obtain the first initial word vector for each of the first words, and obtain the fourth semantic element vector for each of the first words respectively; The first initial word vector, the fourth semantic element vector, the sentence identifier, and the position information of each first word are superimposed to obtain the fourth word vector corresponding to each first word; The text to be classified is classified based on the fourth word vector, the first vector, and the second vector of each first word to obtain the second category label to which the text to be classified belongs; The target category of the text to be classified is obtained by calculating the weighted sum of the first category label and the second category label.

4. The text classification method according to claim 3, characterized in that, The specific implementation methods for classifying the text to be classified based on the fourth word vector of the first word, the first vector, and the second vector to obtain the second category label to which the text to be classified belongs are as follows: Calculate the fourth distance between the fourth word vector and the first vector; If the fourth distance is less than the fourth threshold, calculate the fifth distance between the fourth word vector and the second vector; If the fifth distance is less than the fifth threshold, the second category label to which the text to be classified belongs is determined as the secondary label; If the fifth distance is not less than the fifth threshold, the second category label to which the text to be classified belongs is determined to be the first-level label.

5. The text classification method according to claim 1, characterized in that, The step of determining the main constituent words of the text to be classified from the grammatical relationships between the first words in the text to be classified includes: Syntactic analysis is performed on the grammatical relationships between the first words in the text to be classified to obtain the grammatical structure information of the text to be classified. The text to be classified is then input into a classic automatic text summarization model for summary extraction to obtain a summary of the text to be classified. The grammatical structure information indicates the part of speech of each first word. Based on the grammatical structure information, the modifiers in the modifier template library are searched from the summary to obtain the modifiers of the text to be classified. The modifier template library is pre-built. The modifiers of the text to be classified are removed to obtain text labels; The text labels and the text to be classified are input into a pre-trained language model to obtain the principal component words of the text to be classified. The pre-trained language model is used to perform principal component analysis on the text to be classified using the text labels to obtain the principal component words of the text to be classified.

6. The text classification method according to claim 5, characterized in that, The specific process of obtaining the pre-trained language model includes: Obtain text samples; Syntactic analysis is performed on the grammatical relationships between the second words in the text sample to obtain the sample grammatical structure information of the text sample, and the sample grammatical structure information indicates the part of speech of each second word; The text sample is input into a classic automatic text summarization model for summary extraction to obtain a summary of the text sample; Based on the sample grammatical structure information, the modifiers in the summary are deleted to obtain sample tags, wherein the modifiers are at least one of the pre-constructed modifier templates; The sample labels and the text samples are input into the pre-trained language model to be trained, and the pre-trained language model is obtained.

7. The text classification method according to claim 1, characterized in that, The text to be classified is the dialogue text between a chatbot and a user, which includes business information inquired by the user. The first category label is the business category label to which the business information belongs. After determining the first category label to which the text sentence belongs based on the first semantic vector from the first-level label and the second-level label, the method further includes: Based on the business category tags, the system recommends target businesses related to the business category tags to the user.

8. A text classification device, characterized in that, include: The acquisition module is used to acquire the text to be classified, which carries a primary tag and a secondary tag, wherein the secondary tag is a sub-tag of the primary tag; The word segmentation module is used to segment the text to be classified into words to obtain multiple first words corresponding to the text to be classified. The determining module is used to determine the main component words of the text to be classified from the text to be classified based on the grammatical relationship between each first word in the text to be classified, wherein the main component words are at least one of the first words; The acquisition module is further configured to acquire the first semantic element vector of the main component word, wherein the first semantic element vector indicates the semantics of the main component word; The determining module is further configured to determine the first category label to which the text to be classified belongs based on the first semantic vector from the first-level label and the second-level label; the determining unit is specifically configured to: Obtain the first word vector of the principal component words; Obtain the second word vector and the second semantic element vector of the first-level tag, as well as the third word vector and the third semantic element vector of the second-level tag, wherein the second semantic element vector indicates the semantics of the first-level tag, and the third semantic element vector indicates the semantics of the second-level tag; The first vector of the first-level label is determined based on the second word vector and the second semantic element vector, and the second vector of the second-level label is determined based on the third word vector and the third semantic element vector; The text to be classified is classified based on the first word vector, the first semantic element vector, the first vector, and the second vector to obtain the first category label.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the text classification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the text classification method as described in any one of claims 1 to 7.

Images