Text classification model training method, text classification method and related equipment

By training the encoder and decoder of the text classification model, using training text and tags to expand information, the problem of low text classification efficiency in the existing technology is solved, and more accurate text matching with tags is achieved, and classification efficiency is improved.

CN119025674APending Publication Date: 2024-11-26CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410994764.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing deep learning models cannot accurately match the correlation between text content and labels, resulting in low text classification efficiency.

Method used

By training the encoder and decoder in the text classification model, using the training text, labels and their augmented information, the features are extracted and matching labels are determined in the candidate tag set, improving classification efficiency.

Benefits of technology

It improves the efficiency of text classification, can match text and labels more accurately, and improves the processing ability of massive labels.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025674A_ABST
    Figure CN119025674A_ABST
Patent Text Reader

Abstract

The invention provides a training method of a text classification model, a text classification method and related equipment, the text classification model comprises an encoder and a decoder, and the method comprises the following steps: obtaining a training data set, the training data set comprises a training text, a first label corresponding to the training text and label expansion information corresponding to the first label, and the label expansion information is description information of the content of the first label; and training the text classification model to obtain a target text classification model. According to the method, the target text classification model is obtained by training the encoder and the decoder in the text classification model by using the training text, the first label corresponding to the training text and the label expansion information corresponding to the first label, so that the text classification efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular to a text classification model training method, a text classification method, and related equipment. Background Art

[0002] With the rapid development of the Internet and the rapid progress of information technology, a huge amount of text data has been created and accumulated, covering various fields and topics. Text data classification is generally achieved through labels. How to automatically discover and extract multiple relevant labels from a large number of labels, and how to deal with the correlation and degree of correlation between labels have become the key issues of label recognition. At present, text is generally recognized through deep learning models, but the existing deep learning models cannot accurately match the correlation between text content and labels, resulting in low efficiency in identifying labels corresponding to text. Summary of the invention

[0003] The embodiments of the present application provide a text classification model training method, a text classification method and related equipment to solve the problem of low text classification efficiency in the prior art.

[0004] To solve the above problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for training a text classification model, wherein the text classification model includes an encoder and a decoder, and the method includes:

[0006] Acquire a training data set, the training data set including a training text, a first label corresponding to the training text, and label extension information corresponding to the first label, the label extension information being description information of the first label content;

[0007] Training the text classification model to obtain a target text classification model, wherein the target text classification model is used to recognize input text and output at least one label matching the input text;

[0008] Wherein, the target text classification model includes a target encoder and a target decoder, the target encoder is an encoder obtained by training the encoder based on the training data set, and the target encoder is used to extract features from input data, wherein the input data includes: text and labels; the target decoder is a decoder obtained by training the decoder based on the training data set, and the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, and the candidate label set includes multiple candidate labels.

[0009] Optionally, the training of the text classification model to obtain a target text classification model includes:

[0010] Performing text feature extraction on the training text to obtain a plurality of first text segments;

[0011] respectively calculating the importance scores corresponding to the plurality of first text segments to obtain a plurality of importance scores, wherein the plurality of importance scores correspond one-to-one to the plurality of first text segments;

[0012] Filtering the plurality of first text segments according to the plurality of importance scores to obtain a plurality of second text segments, wherein the plurality of second text segments include first text segments whose importance scores are greater than or equal to a first preset threshold among the plurality of first text segments;

[0013] Calculating the similarity between each second text segment and other second text segments in the plurality of second text segments to obtain a plurality of similarity scores, wherein the plurality of similarity scores correspond to the second text segments one by one;

[0014] The plurality of second text segments are screened and sorted according to the plurality of similarity scores to obtain a plurality of third text segments, wherein the similarity scores corresponding to the plurality of third text segments are sorted from high to low and the similarity scores corresponding to the plurality of third text segments are greater than or equal to a second preset threshold;

[0015] The encoder is trained based on the multiple third text segments, the first label, and label expansion corresponding to the first label to obtain the target encoder.

[0016] Optionally, the extracting text features from the training text to obtain a plurality of first text segments includes:

[0017] A plurality of anchor frames are determined in the training text to obtain a plurality of the first text segments, the anchor frames being used to select text content in the training text, and a ratio of the number of characters in the plurality of first text segments to the number of characters in the training text is greater than or equal to a third preset threshold.

[0018] Optionally, the training of the text classification model to obtain a target text classification model includes:

[0019] Converting the training text into a first vector, and converting the plurality of candidate tags into a plurality of second vectors, wherein the plurality of second vectors correspond one-to-one to the plurality of candidate tags;

[0020] Inputting the first vector and the plurality of second vectors into the target encoder for label decoding to obtain a plurality of similarity values, wherein the plurality of similarities correspond to the plurality of second vectors one-to-one, and the similarities are used for the similarity between the corresponding second vector and the first vector;

[0021] Determine at least one target tag according to the multiple similarity values, the at least one target tag comprising: tags among the multiple candidate tags whose similarity is greater than or equal to a fourth preset threshold;

[0022] The decoder is updated according to a target loss value to obtain the target decoder, where the target loss value is a loss value between the at least one target label and the first label.

[0023] In a second aspect, an embodiment of the present application provides a text classification method, which is applied to a target text classification model, wherein the target text classification model includes a target encoder and a target decoder, and the method includes:

[0024] Get the text to be recognized;

[0025] Inputting the text to be recognized into the target encoder for feature extraction to obtain multiple target text features;

[0026] The multiple target text features are input into the target decoder for classification with a candidate tag set, and at least one candidate tag is output. The at least one candidate tag is a tag that matches the text to be recognized, and the candidate tag set includes multiple candidate tags.

[0027] Optionally, before obtaining the text to be recognized, the method further includes:

[0028] Obtaining a tag deletion request, where the tag deletion request is used to delete a third tag in the candidate tag set;

[0029] The third tag is deleted from the candidate tag set based on the tag deletion request.

[0030] Optionally, before obtaining the text to be recognized, the method further includes:

[0031] Obtain a tag adding request, where the tag adding request is used to request adding a fourth tag to the candidate tag set;

[0032] Acquire a plurality of fourth text segments corresponding to the fourth tag according to the fourth tag, and acquire a tag expansion corresponding to the fourth tag;

[0033] Generate a tag semantic vector according to the fourth tag, the plurality of fourth text segments, and tag expansion corresponding to the fourth tag;

[0034] The tag semantic vector is added to the candidate tag set.

[0035] In a third aspect, an embodiment of the present application further provides a training device for a text classification model, wherein the text classification model includes an encoder and a decoder, and the device includes:

[0036] A first acquisition module is used to acquire a training data set, wherein the training data set includes a training text, a first label corresponding to the training text, and label extension information corresponding to the first label, wherein the label extension information is description information of the first label content;

[0037] A training module, used for training the text classification model to obtain a target text classification model, wherein the target text classification model is used for recognizing input text and outputting at least one label matching the input text;

[0038] Wherein, the target text classification model includes a target encoder and a target decoder, the target encoder is an encoder obtained by training the encoder based on the training data set, and the target encoder is used to extract features from input data, wherein the input data includes: text and labels; the target decoder is a decoder obtained by training the decoder based on the training data set, and the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, and the candidate label set includes multiple candidate labels.

[0039] In a fourth aspect, an embodiment of the present application further provides a text classification device, which is applied to a target text classification model, wherein the target text classification model includes a target encoder and a target decoder, and the device includes:

[0040] The second acquisition module is used to acquire the text to be recognized;

[0041] An extraction module, used for inputting the text to be recognized into the target encoder for feature extraction to obtain a plurality of target text features;

[0042] A classification module is used to input the multiple target text features into the target decoder for classification with the candidate tag set, and output at least one candidate tag, wherein the at least one candidate tag is a tag that matches the text to be recognized, and the candidate tag set includes multiple candidate tags.

[0043] In the fifth aspect, an embodiment of the present application also provides a communication device, comprising: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the method described in the first aspect above; or, the steps in the method described in the second aspect above.

[0044] In a sixth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, wherein when the program is executed by a processor, the program implements the steps in the method described in the first aspect above, or implements the steps in the method described in the second aspect above.

[0045] In the seventh aspect, an embodiment of the present application also provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps in the method described in the first aspect, or to implement the steps in the method described in the second aspect.

[0046] The present application provides a training method for a text classification model, a text classification method and related equipment, wherein the text classification model includes an encoder and a decoder, and the method includes: obtaining a training data set, wherein the training data set includes training text, a first label corresponding to the training text and label expansion information corresponding to the first label, wherein the label expansion information is description information of the content of the first label; training the text classification model to obtain a target text classification model, wherein the target text classification model is used to identify input text and output at least one label matching the input text; wherein the target text classification model includes a target encoder and a target decoder, wherein the target encoder is an encoder obtained by training the encoder based on the training data set, wherein the target encoder is used to perform feature extraction on the input data, wherein the input data includes: text and labels; wherein the target decoder is a decoder obtained by training the decoder based on the training data set, wherein the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, wherein the candidate label set includes a plurality of candidate labels. The present application obtains a target text classification model by training the encoder and decoder in the text classification model using training text, the first label corresponding to the training text, and the label expansion information corresponding to the first label, thereby improving the efficiency of text classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 It is a flowchart of a training method for a text classification model provided in an embodiment of the present application;

[0049] Figure 2 is a schematic diagram of training an encoder according to an embodiment of the present application;

[0050] Figure 3 is a schematic diagram of extracting text features from the training text in an embodiment of the present application;

[0051] Figure 4 This is one of the schematic diagrams of training a decoder in an embodiment of the present application;

[0052] Figure 5 This is the second schematic diagram of training a decoder in an embodiment of the present application;

[0053] Figure 6 It is a flowchart of a text classification method provided in an embodiment of the present application;

[0054] Figure 7 It is a structural schematic diagram of a training device for a text classification model provided in an embodiment of the present application;

[0055] Figure 8 is a structural diagram of a text classification device provided in an embodiment of the present application;

[0056] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] The terms "first", "second" etc. in the embodiments of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of connected objects, such as A and / or B and / or C, representing the inclusion of separate A, separate B, separate C, and A and B all exist, B and C all exist, A and C all exist, and 7 situations that A, B and C all exist.

[0059] See also Figure 1 , Figure 1 It is a flowchart of the training method of the text classification model provided in the embodiment of the present application. Figure 1 The text classification model shown includes an encoder and a decoder, and the method includes:

[0060] Step 101: Acquire a training data set, wherein the training data set includes a training text, a first tag corresponding to the training text, and tag extension information corresponding to the first tag, wherein the tag extension information is description information of the first tag content.

[0061] In this embodiment, the text classification model is mainly composed of an encoder and a decoder, wherein, in the training process of the text classification model, the parameters of the encoder and the decoder are mainly trained respectively. Among them, the training data includes the training text, the first label of the training text and the label expansion information corresponding to the first label, wherein the first label is the classification content of the training text, such as "football". It should be noted that each training text can correspond to multiple labels. For multi-label classification, the same text to be classified may have multiple labels. Therefore, when actually selecting a label, it is necessary to randomly select a label as a training sample. The label is generally short, such as "football". When actually inputting, it is necessary to explain the corresponding meaning of the label, and the input of the label content, i.e., the label expansion information, of the related description synchronously, such as "football: football is a sports event that mainly uses the foot to control and dominate the ball. Two teams attack and defend each other on the same rectangular field according to certain rules. Because football is highly confrontational, has changeable tactics, and has a large number of participants, it is called "the world's first sport". The expanded content can be directly used as the input of the encoder.

[0062] Step 102: training the text classification model to obtain a target text classification model, wherein the target text classification model is used to recognize input text and output at least one label matching the input text;

[0063] Wherein, the target text classification model includes a target encoder and a target decoder, the target encoder is an encoder obtained by training the encoder based on the training data set, and the target encoder is used to extract features from input data, wherein the input data includes: text and labels; the target decoder is a decoder obtained by training the decoder based on the training data set, and the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, and the candidate label set includes multiple candidate labels.

[0064] In this embodiment, when training the text classification model, it is necessary to train the encoder to obtain the target encoder and the decoder to obtain the target decoder. Specifically, in this embodiment, the target encoder mainly performs feature extraction on the input data. Specifically, the input data may be text and / or labels, thereby extracting features of the text / or labels, and training the encoder to obtain the target encoder. At the same time, the encoder can realize semantic feature extraction of labels and texts.

[0065] The target decoder is used to determine at least one candidate tag matching the training text in the candidate tag set, wherein the candidate tag set includes a plurality of candidate tags, and each candidate tag has different corresponding content.

[0066] The present application provides a training method for a text classification model, the text classification model includes an encoder and a decoder, the method includes: obtaining a training data set, the training data set includes a training text, a first label corresponding to the training text, and label expansion information corresponding to the first label, the label expansion information is description information of the first label content; training the text classification model to obtain a target text classification model, the target text classification model is used to identify input text and output at least one label matching the input text; wherein the target text classification model includes a target encoder and a target decoder, the target encoder is an encoder obtained by training the encoder based on the training data set, the target encoder is used to extract features from the input data, wherein the input data includes: text and labels; the target decoder is a decoder obtained by training the decoder based on the training data set, the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, the candidate label set includes a plurality of the candidate labels. The present application obtains a target text classification model by training the encoder and the decoder in the text classification model using the training text, the first label corresponding to the training text, and the label expansion information corresponding to the first label, thereby improving the classification efficiency of the text.

[0067] In some feasible implementations, optionally, the training of the text classification model to obtain a target text classification model includes:

[0068] Performing text feature extraction on the training text to obtain a plurality of first text segments;

[0069] respectively calculating the importance scores corresponding to the plurality of first text segments to obtain a plurality of importance scores, wherein the plurality of importance scores correspond one-to-one to the plurality of first text segments;

[0070] Filtering the plurality of first text segments according to the plurality of importance scores to obtain a plurality of second text segments, wherein the plurality of second text segments include first text segments whose importance scores are greater than or equal to a first preset threshold among the plurality of first text segments;

[0071] Calculating the similarity between each second text segment and other second text segments in the plurality of second text segments to obtain a plurality of similarity scores, wherein the plurality of similarity scores correspond to the second text segments one by one;

[0072] The plurality of second text segments are screened and sorted according to the plurality of similarity scores to obtain a plurality of third text segments, wherein the similarity scores corresponding to the plurality of third text segments are sorted from high to low and the similarity scores corresponding to the plurality of third text segments are greater than or equal to a second preset threshold;

[0073] The encoder is trained based on the multiple third text segments, the first label, and label expansion corresponding to the first label to obtain the target encoder.

[0074] In this embodiment, if Figure 2 As shown, it is a schematic diagram of training the encoder to obtain the target encoder, wherein the encoder can accept two different inputs, including the text content to be classified and the label content, and realizes the alignment of the text to be classified and the label content text in the semantic space through contrastive learning; the specific training steps are as follows: text feature extraction is performed on the training text to obtain multiple first text fragments, specifically, the text feature fragment extraction can reduce the interference of dirty data in label classification (such as: embedded with irrelevant short sentences, additional related links / news, etc.).

[0075] The importance scores corresponding to the multiple first text segments are calculated respectively to obtain multiple importance scores, and the multiple first text segments are screened according to the multiple importance scores to obtain multiple second text segments, wherein the multiple second text segments include the first text segments whose importance scores are greater than or equal to the first preset threshold value among the multiple first text segments. Specifically, the importance of the text content in the segment is calculated (for example, the tfidf algorithm can be used, the higher the idf, the greater the content value of the information), so as to screen out segments with less information / unimportant segments.

[0076] Calculate the similarity between each second text segment and other second text segments in the multiple second text segments to obtain multiple similarity scores, and the multiple similarity scores correspond to the second text segments one by one; screen and sort the multiple second text segments according to the multiple similarity scores to obtain multiple third text segments, and the similarity score values ​​corresponding to the multiple third text segments are sorted from high to low and the similarity scores corresponding to the multiple third text segments are greater than or equal to a second preset threshold; train the encoder based on the multiple third text segments, the first label and the label expansion corresponding to the first label to obtain the target encoder. Specifically, construct a segment content relevance matrix MS, where MSij represents the relevance of segments Si, Sj. Before model training, word2vec / SimBERT can be used to directly obtain the semantic representation of the segment to calculate the relevance between segments. After the model training is completed, the trained model can be used to obtain the semantic relevance of the segment. It can be expressed here as follows:

[0077] MS ij =Cosine(Encoder(Si),Encoder(Sj))

[0078] Through the edge probability, for each segment, the relevance of the segment to other segments can be calculated and used as the relevance score; by sorting {S1, S2, ..., Sn} in descending order of relevance scores, the relevance scores are sequentially accumulated. When the accumulated relevance score is less than 80% of the total score, it is considered a valid segment, and the other segments are invalid segments. The final valid relevance score segment is {S1, S2, ..., Sm}. Finally, the valid segments are directly spliced ​​according to the relevance order of the segments as the input of the encoder.

[0079] Optionally, the extracting text features from the training text to obtain a plurality of first text segments includes:

[0080] A plurality of anchor frames are determined in the training text to obtain a plurality of the first text segments, the anchor frames being used to select text content in the training text, and a ratio of the number of characters in the plurality of first text segments to the number of characters in the training text is greater than or equal to a third preset threshold.

[0081] In this embodiment, if Figure 3 As shown, Figure 3 This is a schematic diagram of extracting text features from the training text in this embodiment. Specifically, an anchor frame fragment is randomly extracted: a starting point position is randomly selected, the size of the corresponding offset is randomly selected (generally 16, 32, 64, 128 can be selected) or a corresponding offset frame (anchor frame, the frame size can be 1 / 10, 1 / 5 of the total size, etc.) is randomly generated, and the text in the frame is represented by S.

[0082] Therefore, the anchor frame segments are repeatedly extracted: if the sum of the text lengths in the current S-frame set is greater than 80% of the total text length or greater than a threshold (such as 1024, etc.), the extraction is stopped, otherwise the previous step is repeated. At this time, the corresponding text is obtained as the extracted text segment, which can be expressed as {s1, s2, s3, ..., sn}.

[0083] By extracting features from the text to be classified through anchor boxes, the data extraction efficiency can be effectively improved, thereby reducing the interference of redundant data.

[0084] Optionally, the training of the text classification model to obtain a target text classification model includes:

[0085] Converting the training text into a first vector, and converting the plurality of candidate tags into a plurality of second vectors, wherein the plurality of second vectors correspond one-to-one to the plurality of candidate tags;

[0086] Inputting the first vector and the plurality of second vectors into the target encoder for label decoding to obtain a plurality of similarity values, wherein the plurality of similarities correspond to the plurality of second vectors one-to-one, and the similarities are used for the similarity between the corresponding second vector and the first vector;

[0087] Determine at least one target tag according to the multiple similarity values, the at least one target tag comprising: tags among the multiple candidate tags whose similarity is greater than or equal to a fourth preset threshold;

[0088] The decoder is updated according to a target loss value to obtain the target decoder, where the target loss value is a loss value between the at least one target label and the first label.

[0089] In this embodiment, if Figure 4 and Figure 5 As shown in FIG. 1 , a schematic diagram of training the decoder to obtain the target decoder is shown. The decoder training depends on the encoder having completed the training. By encoding all labels and expressions, a vector representation (L, {l1, l2, l3, ...}) of all label contents can be obtained, which can be directly stored in the vector database for future use.

[0090] Input the text to be classified, and obtain the feature H of the text to be classified through the encoder; based on the feature H of the text to be classified, enter the tag set L to perform similarity vector retrieval, and obtain the candidate tags {l1,l2,l3,...,l1} with a similarity greater than the threshold / directly select the TopK tags with the highest similarity k}; The candidate tag set is concatenated as an input to the decoder in the following way, where " <sep>"To separate, add a " <bos>” label, indicating the official start of outputting the final label results.

[0091] When decoding labels, conventional decoders use decoding based on the label space, that is, the decoded implicit vector. This means that if new labels are added, the label space cannot be expanded quickly. Once expanded, the model needs to be retrained. Here, word-based decoding is used directly, and in order to ensure the controllability of word decoding, the input prompt is limited, and the vocabulary is further controlled, so that controllable generation of labels can be achieved.

[0092] The actual decoding process is as follows: construct a candidate output dictionary and segment the concatenated labels, such as " <sep>football <sep>basketball <sep>Tennis can be divided into <sep>", "foot", "ball", <sep>","basketball"," <sep>", "net", "ball", build Mask word ={0,0,0,1,0,....,0,0}, the dimension size is the actual total word list size, where the position where the concatenated word exists is 1, and the other positions can be directly set to zero, and increase" <eos>"The end flag is also set to 1.

[0093] When the input is " <bos>"Before, h x =Decoder(h x-1 ,P x ), where h x The hidden variable representing the current state, P x Represents the prompt feature of the current state input. When x=0, h0=Decoder(P0); when the input is " <bos>"When <bos>< / bos> =Decoder(h x-1 ,P <bos>< / bos> ,h input ), where h input The text feature hidden variable passed by the encoder; when the input is " <bos>"After that, h x =Decoder(h x-1 ,P x ), where h x is the hidden variable of the actual output.

[0094] The actual output sequence probability is: FFN(h x )Mask word Where FFN represents a fully connected network, where the weight matrix size is [W, D], where W represents the vocabulary size and D represents the dimension of the latent variable H; the actual encoder training loss can be expressed as follows, where X is the feature representation, Y is the corresponding actual label, and its sequence construction method is consistent with the above method, and P is the feature representation of the relevant label retrieved based on X: Loss = CrossEntroyLoss(FFN(Decoder(Encoder(X),P))Mask word , Y).

[0095] The present application obtains a target text classification model by training the encoder and decoder in the text classification model using training text, the first label corresponding to the training text, and the label expansion information corresponding to the first label, thereby improving the efficiency of text classification.

[0096] See also Figure 6 , Figure 6 : is a flow chart of a text classification method provided in an embodiment of the present application, which is applied to a target text classification model, wherein the target text classification model includes a target encoder and a target decoder, and the method includes:

[0097] Step 601: Obtain text to be recognized.

[0098] Step 602: Input the text to be recognized into the target encoder for feature extraction to obtain multiple target text features.

[0099] Step 603: Input the multiple target text features into the target decoder for classification with the candidate tag set, and output at least one candidate tag, wherein the at least one candidate tag is a tag that matches the text to be recognized, and the candidate tag set includes multiple candidate tags.

[0100] In this embodiment, the target text classification model refers to the content in the above embodiment, that is, the target text classification model is used to classify the input content, specifically, to assign at least one candidate label to the input content.

[0101] In some feasible implementation manners, optionally, before obtaining the text to be recognized, the method further includes:

[0102] Obtaining a tag deletion request, where the tag deletion request is used to delete a third tag in the candidate tag set;

[0103] The third tag is deleted from the candidate tag set based on the tag deletion request.

[0104] In this embodiment, on the basis of completing the training and reasoning process of the full model, it is possible to further realize the dynamic processing of new labels (including deletion and addition of labels) without retraining the original model, which is very meaningful for the scenario of massive labels. Specifically, after receiving a label deletion request, since the label has no direct relationship with the FFN layer, it only needs to be deleted directly in the vector database, and the corresponding label will no longer be retrieved in the actual reasoning stage. Therefore, combined with controllable label generation, the corresponding label will no longer be predicted.

[0105] Optionally, before obtaining the text to be recognized, the method further includes:

[0106] Obtain a tag adding request, where the tag adding request is used to request adding a fourth tag to the candidate tag set;

[0107] Acquire a plurality of fourth text segments corresponding to the fourth tag according to the fourth tag, and acquire a tag expansion corresponding to the fourth tag;

[0108] Generate a tag semantic vector according to the fourth tag, the plurality of fourth text segments, and tag expansion corresponding to the fourth tag;

[0109] The tag semantic vector is added to the candidate tag set.

[0110] In this embodiment, when adding a new tag, it is only necessary to regenerate the feature vector of the new tag. The actual steps of adding a new tag are as follows: obtain M pieces of text content corresponding to the new tag (recommended> 10), and obtain its corresponding text semantic representation {h1,h2,h3,...,h m }; Describe the tag content in detail and obtain its corresponding semantic representation h label ; Directly obtain the average value of the semantic representation of the label corresponding text and label description as the final label semantic vector: H Label =(h1+h2+h3+...+h m ) / m+h label ) / 2; Add the new semantic tag to the semantic tag library to complete the tag addition.

[0111] In the scenario where the candidate tag set is a massive set of tags, the decoding structure is separated during the tag recognition process, and semantic retrieval and word-based controllable tag generation methods are used at the same time, so that the tag set can be quickly added or deleted without retraining the text classification model.

[0112] This application is applied to Mobile Cloud - Natural Language Processing Basic Services: Text classification belongs to the interface service corresponding to this product, and can also be applied to the technical middle platform - AI key core technology research: Natural language understanding - text classification and named entity recognition capability cluster results include text classification capabilities, which can simplify the difficulty of video retrieval and improve user experience. Compared with the current massive video analysis scenarios, it has very high value.

[0113] The present application obtains a target text classification model by training the encoder and decoder in the text classification model using training text, the first label corresponding to the training text, and the label expansion information corresponding to the first label, thereby improving the efficiency of text classification.

[0114] See also Figure 7 , Figure 7 is a structural diagram of a training device for a text classification model provided in an embodiment of the present application. Figure 7 As shown, the text classification model includes an encoder and a decoder, and the training device 700 of the text classification model includes:

[0115] A first acquisition module 710 is used to acquire a training data set, wherein the training data set includes a training text, a first tag corresponding to the training text, and tag extension information corresponding to the first tag, wherein the tag extension information is description information of the first tag content;

[0116] A training module 720 is used to train the text classification model to obtain a target text classification model, wherein the target text classification model is used to recognize input text and output at least one label matching the input text;

[0117] Wherein, the target text classification model includes a target encoder and a target decoder, the target encoder is an encoder obtained by training the encoder based on the training data set, and the target encoder is used to extract features from input data, wherein the input data includes: text and labels; the target decoder is a decoder obtained by training the decoder based on the training data set, and the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, and the candidate label set includes multiple candidate labels.

[0118] Optionally, the training module 720 includes:

[0119] An extraction submodule, used for performing text feature extraction on the training text to obtain a plurality of first text segments;

[0120] A first calculation submodule, configured to respectively calculate importance scores corresponding to the plurality of first text segments to obtain a plurality of importance scores, wherein the plurality of importance scores correspond one-to-one to the plurality of first text segments;

[0121] a screening submodule, configured to screen the plurality of first text segments according to the plurality of importance scores to obtain a plurality of second text segments, wherein the plurality of second text segments include a first text segment having an importance score greater than or equal to a first preset threshold value among the plurality of first text segments;

[0122] A second calculation submodule, configured to calculate the similarity between each second text segment and other second text segments in the plurality of second text segments to obtain a plurality of similarity scores, wherein the plurality of similarity scores correspond one-to-one to the second text segments;

[0123] a sorting submodule, configured to screen and sort the plurality of second text segments according to the plurality of similarity scores to obtain a plurality of third text segments, wherein the similarity scores corresponding to the plurality of third text segments are sorted from high to low and the similarity scores corresponding to the plurality of third text segments are greater than or equal to a second preset threshold;

[0124] A training submodule is used to train the encoder based on the multiple third text segments, the first label and the label expansion corresponding to the first label to obtain the target encoder.

[0125] Optionally, the extraction submodule includes:

[0126] A determination unit is used to determine multiple anchor boxes in the training text to obtain multiple first text fragments, the anchor boxes are used to select text content in the training text, and the ratio of the number of characters in the multiple first text fragments to the number of characters in the training text is greater than or equal to a third preset threshold.

[0127] The present application obtains a target text classification model by training the encoder and decoder in the text classification model using training text, the first label corresponding to the training text, and the label expansion information corresponding to the first label, thereby improving the efficiency of text classification.

[0128] See also Figure 8 , Figure 8 is a structural diagram of a text classification device provided in an embodiment of the present application. Figure 8 As shown, it is applied to a target text classification model, the target text classification model includes a target encoder and a target decoder, and the text classification device 800 includes:

[0129] The second acquisition module 810 is used to acquire the text to be recognized;

[0130] An extraction module 820 is used to input the text to be recognized into the target encoder for feature extraction to obtain multiple target text features;

[0131] The classification module 830 is used to input the multiple target text features into the target decoder to classify with the candidate tag set, and output at least one candidate tag, wherein the at least one candidate tag is a tag that matches the text to be recognized, and the candidate tag set includes multiple candidate tags.

[0132] Optionally, also include:

[0133] A third acquisition module, used to acquire a tag deletion request, where the tag deletion request is used to delete a third tag in the candidate tag set;

[0134] The deleting module is configured to delete the third tag from the candidate tag set based on the tag deletion request.

[0135] Optionally, also include:

[0136] A fourth acquisition module, used for acquiring a tag adding request, where the tag adding request is used for requesting to add a fourth tag to the candidate tag set;

[0137] a fifth acquisition module, configured to acquire, according to the fourth tag, a plurality of fourth text segments corresponding to the fourth tag, and acquire a tag expansion corresponding to the fourth tag;

[0138] A generating module, configured to generate a tag semantic vector according to the fourth tag, the plurality of fourth text segments and a tag expansion corresponding to the fourth tag;

[0139] An adding module is used to add the tag semantic vector to the candidate tag set.

[0140] The present application obtains a target text classification model by training the encoder and decoder in the text classification model using training text, the first label corresponding to the training text, and the label expansion information corresponding to the first label, thereby improving the efficiency of text classification.

[0141] The present application also provides an electronic device. Fig. 9 , the electronic device may include a processor 901, a memory 902, and a program 9021 stored in the memory 902 and executable on the processor 901.

[0142] When program 9021 is executed by processor 901, it can achieve Figure 1 or Figure 6 Any step in the corresponding method embodiment:

[0143] Acquire a training data set, the training data set including a training text, a first label corresponding to the training text, and label extension information corresponding to the first label, the label extension information being description information of the first label content;

[0144] Training the text classification model to obtain a target text classification model, wherein the target text classification model is used to recognize input text and output at least one label matching the input text;

[0145] Wherein, the target text classification model includes a target encoder and a target decoder, the target encoder is an encoder obtained by training the encoder based on the training data set, and the target encoder is used to extract features from input data, wherein the input data includes: text and labels; the target decoder is a decoder obtained by training the decoder based on the training data set, and the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, and the candidate label set includes multiple candidate labels.

[0146] Optionally, the training of the text classification model to obtain a target text classification model includes:

[0147] Performing text feature extraction on the training text to obtain a plurality of first text segments;

[0148] respectively calculating the importance scores corresponding to the plurality of first text segments to obtain a plurality of importance scores, wherein the plurality of importance scores correspond one-to-one to the plurality of first text segments;

[0149] Filtering the plurality of first text segments according to the plurality of importance scores to obtain a plurality of second text segments, wherein the plurality of second text segments include first text segments whose importance scores are greater than or equal to a first preset threshold among the plurality of first text segments;

[0150] Calculating the similarity between each second text segment and other second text segments in the plurality of second text segments to obtain a plurality of similarity scores, wherein the plurality of similarity scores correspond to the second text segments one by one;

[0151] The plurality of second text segments are screened and sorted according to the plurality of similarity scores to obtain a plurality of third text segments, wherein the similarity scores corresponding to the plurality of third text segments are sorted from high to low and the similarity scores corresponding to the plurality of third text segments are greater than or equal to a second preset threshold;

[0152] The encoder is trained based on the multiple third text segments, the first label, and label expansion corresponding to the first label to obtain the target encoder.

[0153] Optionally, the extracting text features from the training text to obtain a plurality of first text segments includes:

[0154] A plurality of anchor frames are determined in the training text to obtain a plurality of the first text segments, the anchor frames being used to select text content in the training text, and a ratio of the number of characters in the plurality of first text segments to the number of characters in the training text is greater than or equal to a third preset threshold.

[0155] Optionally, the training of the text classification model to obtain a target text classification model includes:

[0156] Converting the training text into a first vector, and converting the plurality of candidate tags into a plurality of second vectors, wherein the plurality of second vectors correspond one-to-one to the plurality of candidate tags;

[0157] Inputting the first vector and the plurality of second vectors into the target encoder for label decoding to obtain a plurality of similarity values, wherein the plurality of similarities correspond to the plurality of second vectors one-to-one, and the similarities are used for the similarity between the corresponding second vector and the first vector;

[0158] Determine at least one target tag according to the multiple similarity values, the at least one target tag comprising: tags among the multiple candidate tags whose similarity is greater than or equal to a fourth preset threshold;

[0159] The decoder is updated according to a target loss value to obtain the target decoder, where the target loss value is a loss value between the at least one target label and the first label.

[0160] or

[0161] Get the text to be recognized;

[0162] Inputting the text to be recognized into the target encoder for feature extraction to obtain multiple target text features;

[0163] The multiple target text features are input into the target decoder for classification with a candidate tag set, and at least one candidate tag is output. The at least one candidate tag is a tag that matches the text to be recognized, and the candidate tag set includes multiple candidate tags.

[0164] Optionally, before obtaining the text to be recognized, the method further includes:

[0165] Obtaining a tag deletion request, where the tag deletion request is used to delete a third tag in the candidate tag set;

[0166] The third tag is deleted from the candidate tag set based on the tag deletion request.

[0167] Optionally, before obtaining the text to be recognized, the method further includes:

[0168] Obtain a tag adding request, where the tag adding request is used to request adding a fourth tag to the candidate tag set;

[0169] Acquire a plurality of fourth text segments corresponding to the fourth tag according to the fourth tag, and acquire a tag expansion corresponding to the fourth tag;

[0170] Generate a tag semantic vector according to the fourth tag, the plurality of fourth text segments, and tag expansion corresponding to the fourth tag;

[0171] The tag semantic vector is added to the candidate tag set.

[0172] The present application obtains a target text classification model by training the encoder and decoder in the text classification model using training text, the first label corresponding to the training text, and the label expansion information corresponding to the first label, thereby improving the efficiency of text classification.

[0173] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the training method of the text classification module or each process of the text classification method embodiment described above can be implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0174] The embodiments of the present application further provide a computer program product, which is stored in a storage medium. The computer program product is executed by at least one processor to implement the training method of the above-mentioned text classification module or the various processes of the text classification method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0175] It should be noted that, in this article, the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0176] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0177] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.< / bos> < / bos> < / bos> < / eos> < / sep> < / sep> < / sep> < / sep> < / sep> < / sep> < / bos> < / sep>

Claims

1. A training method for a text classification model, characterized in that: The text classification model includes an encoder and a decoder, and the method includes: Acquire a training data set, the training data set including a training text, a first label corresponding to the training text, and label extension information corresponding to the first label, the label extension information being description information of the first label content; Training the text classification model to obtain a target text classification model, wherein the target text classification model is used to recognize input text and output at least one label matching the input text; Wherein, the target text classification model includes a target encoder and a target decoder, the target encoder is an encoder obtained by training the encoder based on the training data set, and the target encoder is used to extract features from input data, wherein the input data includes: text and labels; the target decoder is a decoder obtained by training the decoder based on the training data set, and the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, and the candidate label set includes multiple candidate labels.

2. The method according to claim 1, characterized in that: The training of the text classification model to obtain a target text classification model includes: Performing text feature extraction on the training text to obtain a plurality of first text segments; respectively calculating the importance scores corresponding to the plurality of first text segments to obtain a plurality of importance scores, wherein the plurality of importance scores correspond one-to-one to the plurality of first text segments; Filtering the plurality of first text segments according to the plurality of importance scores to obtain a plurality of second text segments, wherein the plurality of second text segments include first text segments whose importance scores are greater than or equal to a first preset threshold among the plurality of first text segments; Calculating the similarity between each second text segment and other second text segments in the plurality of second text segments to obtain a plurality of similarity scores, wherein the plurality of similarity scores correspond to the second text segments one by one; The plurality of second text segments are screened and sorted according to the plurality of similarity scores to obtain a plurality of third text segments, wherein the similarity scores corresponding to the plurality of third text segments are sorted from high to low and the similarity scores corresponding to the plurality of third text segments are greater than or equal to a second preset threshold; The encoder is trained based on the multiple third text segments, the first label, and label expansion corresponding to the first label to obtain the target encoder.

3. The method according to claim 2, characterized in that The step of extracting text features from the training text to obtain a plurality of first text segments includes: A plurality of anchor frames are determined in the training text to obtain a plurality of the first text segments, the anchor frames being used to select text content in the training text, and a ratio of the number of characters in the plurality of first text segments to the number of characters in the training text is greater than or equal to a third preset threshold.

4. The method according to claim 1, characterized in that: The training of the text classification model to obtain a target text classification model includes: Converting the training text into a first vector, and converting the plurality of candidate tags into a plurality of second vectors, wherein the plurality of second vectors correspond one-to-one to the plurality of candidate tags; Inputting the first vector and the plurality of second vectors into the target encoder for label decoding to obtain a plurality of similarity values, wherein the plurality of similarities correspond to the plurality of second vectors one-to-one, and the similarities are used for the similarity between the corresponding second vector and the first vector; Determine at least one target tag according to the multiple similarity values, the at least one target tag comprising: tags among the multiple candidate tags whose similarity is greater than or equal to a fourth preset threshold; The decoder is updated according to a target loss value to obtain the target decoder, where the target loss value is a loss value between the at least one target label and the first label.

5. A text classification method, characterized in that: Applied to a target text classification model, the target text classification model includes a target encoder and a target decoder, the method includes: Get the text to be recognized; Inputting the text to be recognized into the target encoder for feature extraction to obtain multiple target text features; The multiple target text features are input into the target decoder for classification with a candidate tag set, and at least one candidate tag is output. The at least one candidate tag is a tag that matches the text to be recognized, and the candidate tag set includes multiple candidate tags.

6. The method according to claim 5, characterized in that Before obtaining the text to be recognized, the method further includes: Obtaining a tag deletion request, where the tag deletion request is used to delete a third tag in the candidate tag set; The third tag is deleted from the candidate tag set based on the tag deletion request.

7. The method according to claim 5, characterized in that Before obtaining the text to be recognized, the method further includes: Obtain a tag adding request, where the tag adding request is used to request adding a fourth tag to the candidate tag set; Acquire a plurality of fourth text segments corresponding to the fourth tag according to the fourth tag, and acquire a tag expansion corresponding to the fourth tag; Generate a tag semantic vector according to the fourth tag, the plurality of fourth text segments, and tag expansion corresponding to the fourth tag; The tag semantic vector is added to the candidate tag set.

8. A training device for a text classification model, characterized in that: The text classification model includes an encoder and a decoder, and the device includes: A first acquisition module is used to acquire a training data set, wherein the training data set includes a training text, a first label corresponding to the training text, and label extension information corresponding to the first label, wherein the label extension information is description information of the first label content; A training module, used for training the text classification model to obtain a target text classification model, wherein the target text classification model is used for recognizing input text and outputting at least one label matching the input text; Wherein, the target text classification model includes a target encoder and a target decoder, the target encoder is an encoder obtained by training the encoder based on the training data set, and the target encoder is used to extract features from input data, wherein the input data includes: text and labels; the target decoder is a decoder obtained by training the decoder based on the training data set, and the target decoder is used to determine at least one candidate label matching the training text in a candidate label set, and the candidate label set includes multiple candidate labels.

9. A text classification device, characterized in that: Applied to a target text classification model, the target text classification model includes a target encoder and a target decoder, and the device includes: The second acquisition module is used to acquire the text to be recognized; An extraction module, used for inputting the text to be recognized into the target encoder for feature extraction to obtain a plurality of target text features; A classification module is used to input the multiple target text features into the target decoder for classification with the candidate tag set, and output at least one candidate tag, wherein the at least one candidate tag is a tag that matches the text to be recognized, and the candidate tag set includes multiple candidate tags.

10. An electronic device comprising: A memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the training method of the text classification model as described in any one of claims 1 to 4, or to implement the steps in the text classification method as described in any one of claims 5 to 7.

11. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the program implements the steps in the training method of the text classification model as described in any one of claims 1 to 4, or implements the steps in the text classification method as described in any one of claims 5 to 7.

12. A computer program product, characterized in that The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps in the training method of the text classification model as described in any one of claims 1 to 4, or to implement the steps in the text classification method as described in any one of claims 5 to 7.

Citation Information

Cited By

  • Multi-modal classification model training method, multi-modal classification method and storage medium

    CN120565114A