Hierarchical label text classification method, system, device and storage medium
By using sequence generation model pre-training, attention mechanism, and time series network, combined with multi-granularity feature extraction and masking operation, the problem of unutilized correlation in traditional hierarchical label classification is solved, achieving efficient and accurate hierarchical label text classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHENGDU UNION BIG DATA TECH CO LTD
- Filing Date
- 2023-02-14
- Publication Date
- 2026-07-03
AI Technical Summary
Traditional hierarchical label text classification tasks fail to fully utilize the correlation between hierarchical labels and the potential constraints of coarse-grained labels on fine-grained labels, resulting in poor classification performance.
A sequence generation model is pre-trained, and attention mechanism and time series network are combined. Hierarchical labels are generated through multi-granularity feature extraction and masking operation, and the label information is associated and the generation process is controlled.
It improves the accuracy and efficiency of hierarchical label text classification, solves the uncontrollability in hierarchical label generation tasks, and enhances classification results.
Smart Images
Figure CN116069931B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of natural language processing technology, and more specifically, to a hierarchical label text classification method, system, device, and storage medium based on multi-granularity feature extraction and label sequence generation. Background Technology
[0002] Text classification is one of the most common applications in natural language processing (NLP). Based on the number of labels corresponding to the data, text classification tasks can be categorized into single-label classification tasks and multi-label classification tasks. Further, multi-label classification tasks can be further divided into parallel label classification tasks and hierarchical label classification tasks based on whether the labels contain hierarchical relationships. In practical applications, most text classification tasks involve hierarchical relationships between labels. For example, in document categorization tasks, each document's labels typically present a "directory" format, containing coarse-grained labels and corresponding fine-grained labels. Therefore, improving the performance of hierarchical label text classification and solving hierarchical label text classification tasks in real-world applications is a pressing issue.
[0003] Traditional hierarchical label text classification tasks are typically treated as multi-level label classification tasks, primarily employing two approaches. One approach, based on task transformation, treats all combinations of parent and child labels as a single category, converting hierarchical label classification into parallel label classification. While considering the relationships between hierarchical labels, this approach suffers from the curse of dimensionality when dealing with a large number of labels. The other approach, based on algorithmic adaptability, improves the algorithm to suit multi-level label classification tasks, using the output layer of a neural network to perform binary classification on each label, ultimately combining all predicted labels as the output. The drawback of this method, which converts hierarchical label classification into parallel label classification, is that it completely ignores the relationships between hierarchical labels. Therefore, it can be seen that traditional hierarchical label text classification tasks do not fully utilize the relationships between hierarchical labels and the potential constraints of coarse-grained labels on fine-grained labels. Summary of the Invention
[0004] This invention provides a hierarchical label text classification method, system, device, and storage medium, which effectively solves the problem of poor hierarchical label classification performance in traditional hierarchical label classification tasks due to the lack of full utilization of the correlation between hierarchical labels and the potential limitation of coarse-grained labels on fine-grained labels.
[0005] In a first aspect, embodiments of the present invention provide a hierarchical label text classification method, the method comprising the following steps:
[0006] (1) Construct a sequence generation model based on the hierarchical label classification task, and pre-train the sequence generation model;
[0007] (2) Input the text data into a pre-trained sequence generation model, which processes the text data as follows:
[0008] (2.1) Multi-granularity text feature vectors are extracted using sequence generation;
[0009] (2.2) An attention mechanism is used to encode the multi-granularity text feature vectors and the predicted upper-level labels to obtain an encoded vector containing upper-level label information.
[0010] (2.3) A time series network is used to decode the encoded vector containing the label information of the previous level in order to predict the label vector of the next level, and the label of the previous level is iteratively updated;
[0011] (2.4) Use mask operations to control the generation of all levels of labels.
[0012] In the above embodiments, by fully considering the correlation between tags and the potential constraints of coarse-grained tags on fine-grained tags, hierarchical tags are generated by sequence generation, which links the tag information together, thereby enabling efficient and accurate hierarchical tag text classification; and by using a masking method to control the generation of hierarchical tags, the uncontrollability in the hierarchical tag generation task is solved.
[0013] As some optional implementations of this application, the sequence generation model adopts a seq2seq structure, mainly including an encoder and a decoder. The encoder converts text data into a hidden vector containing its features, while the decoder converts the hidden vector containing features into the corresponding label.
[0014] As some optional implementations of this application, the process for pre-training the sequence generation model is as follows:
[0015] (1.1) Obtain training data and construct a label vocabulary based on the training data;
[0016] (1.2) Input the tag vocabulary into the sequence generation model, wherein the sequence generation model assigns an initial default vector to each tag in the tag vocabulary;
[0017] (1.3) Input the training data into the sequence generation model, which converts the sentences in the training data into index vectors of keywords.
[0018] In the above embodiments, by pre-training the sequence generation model, i.e. configuring the model accordingly, the sequence generation model can quickly and accurately perform hierarchical label classification according to the rules of the hierarchical label classification task.
[0019] As some optional implementations of this application, the process of extracting multi-granularity text feature vectors using sequence generation is as follows:
[0020] (2.11) Input the text data into several encoding layers of the encoder. The encoding layers encode the text data based on sequence generation to obtain the text feature vector of the keywords.
[0021] (2.12) The text feature vectors of each coding layer are summed to obtain the coding vectors of each coding layer corresponding to the keywords.
[0022] As some optional implementations of this application, the process of using an attention mechanism to encode multi-granularity text feature vectors and predicted parent-level labels to obtain encoded vectors containing parent-level label information is as follows:
[0023] (2.21) Vectorize the upper-level label to obtain the label vector. The upper-level label is predicted by the sequence generation model and iteratively updated.
[0024] (2.22) Value vectors and key vectors are obtained by character-level encoding vector mapping, and corresponding index vectors are obtained by tag vector mapping;
[0025] (2.23) Perform cross-attention calculation based on the index vector and key vector to obtain the attention weight vector;
[0026] (2.24) Normalize the attention weight vector to obtain the normalized weight vector;
[0027] (2.25) The value vector is weighted and summed based on the normalized weight vector to obtain the encoding vector containing the label information of the previous level.
[0028] In the above embodiments, features are extracted and fused from multi-granular information of text data, and attention mechanism is used to obtain characteristic information of keywords, which enriches the information contained in the model's encoding vector and improves the model's classification performance.
[0029] As some optional implementations of this application, a time series network is used to decode the encoded vector containing the label information of the previous level in order to predict the label of the next level, and the process of iteratively updating the label of the previous level is as follows:
[0030] (2.31) Input the encoded vector containing the label information of the previous level into the decoder for decoding to obtain the intermediate vector;
[0031] (2.32) Perform a linear transformation on the intermediate vector to obtain the predicted vector of the next level label, and iteratively update the previous level label based on the predicted next level label until the prediction of all level labels is completed.
[0032] As some optional implementations of this application, the process of controlling the generation of all levels of labels using mask operations is as follows:
[0033] (2.41) Mask a portion of the prediction vectors to obtain the indicator vectors;
[0034] (2.42) Calculate the probability values of all labels based on the prediction vector and the indicator vector.
[0035] In the above embodiments, in order to solve the uncontrollability of the hierarchical label classification task, a masking method is used for control, so that the hierarchical label classification task can be performed accurately.
[0036] In a second aspect, the present invention provides a hierarchical label text classification system and a sequence generation model building unit, wherein the sequence generation model building unit builds a sequence generation model based on a hierarchical label classification task;
[0037] A sequence generation model pre-training unit, which is used to pre-train the sequence generation model;
[0038] A sequence generation model unit is used to input text data into a pre-trained sequence generation model and predict hierarchical labels on the text data.
[0039] The sequence generation model unit includes:
[0040] A text feature vector extraction module, wherein the text feature vector extraction module extracts multi-granular text feature vectors by using a sequence generation method;
[0041] The attention mechanism module uses an attention mechanism to encode multi-granularity text feature vectors and predicted upper-level labels to obtain an encoded vector containing upper-level label information.
[0042] The hierarchical label prediction module uses a time series network to decode the encoded vector containing the label information of the previous level in order to predict the label of the next level, and iteratively updates the label of the previous level.
[0043] A mask operation control module, which uses mask operations to control the generation of all levels of labels.
[0044] In a third aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor performing the hierarchical label text classification method.
[0045] In a fourth aspect, the present invention provides a computer-readable storage medium on which a computer program is stored, the computer program being executed by a processor to implement the hierarchical label text classification method.
[0046] The beneficial effects of this invention are as follows:
[0047] 1. By fully considering the correlation between tags and the potential constraints of coarse-grained tags on fine-grained tags, a sequence generation method is used to generate hierarchical tags, linking tag information together, thereby enabling efficient and accurate hierarchical tag text classification; furthermore, by using masking to control the generation of hierarchical tags, the uncontrollability in hierarchical tag classification tasks is solved.
[0048] 2. By extracting and fusing features from multi-granular information in text data and using an attention mechanism to obtain the characteristic information of keywords, the information contained in the encoding vector of the sequence generation model is enriched, thereby improving the classification performance of the sequence generation model. Attached Figure Description
[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 This is a flowchart illustrating the steps of the hierarchical label text classification method according to an embodiment of the present invention;
[0051] Figure 2 This is a schematic diagram of the hierarchical label structure according to an embodiment of the present invention;
[0052] Figure 3 This is a structural diagram of the hierarchical label sequence generation model according to an embodiment of the present invention;
[0053] Figure 4 This is a structural block diagram of the hierarchical label text classification system according to an embodiment of the present invention. Detailed Implementation
[0054] To better understand the above technical solutions, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solutions of the present invention, rather than limitations on the technical solutions of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
[0055] It should also be understood that, in order to simplify the description of the invention and thus aid in the understanding of at least one embodiment, multiple features may sometimes be grouped into a single embodiment, drawing, or description thereof in the foregoing description of the embodiments of the invention. However, this method of disclosure does not imply that the subject matter of the invention requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of the single embodiment disclosed above.
[0056] Example 1
[0057] This invention provides a hierarchical label text classification method. The method achieves hierarchical label text classification based on multi-granularity feature extraction and label sequence generation. Please refer to [link to relevant documentation]. Figure 1 The method includes the following steps:
[0058] Step (1): Construct a sequence generation model based on the hierarchical label classification task, and pre-train the sequence generation model.
[0059] In this embodiment of the invention, constructing a sequence generation model based on a hierarchical label classification task is equivalent to transforming the hierarchical label classification task into a sequence generation task, which is defined as follows:
[0060]
[0061] Where x refers to text data containing m keywords, and y refers to the set of n tags corresponding to x. Please refer to [link to relevant documentation]. Figure 2 There are a total of three hierarchical labels (not limited to three, set according to the task classification of hierarchical labels), where y0 represents the top-level label, y1 represents the first-level label, y2 represents the second-level label, and so on. Therefore, y... i-1 This represents the label at level i-1.
[0062] Specifically, the hierarchical label classification model generated based on label sequences adopts a seq2seq structure. Please refer to [link / reference]. Figure 3 The hierarchical label classification model consists of an encoder on the left, which encodes text data primarily based on sequence generation; and a decoder on the right, which uses a time series network to predict the next level label starting from the top-level label.
[0063] Specifically, the pre-training process for the sequence generation model is as follows:
[0064] (1.1) Obtain training data and construct a label vocabulary based on the training data.
[0065] (1.2) Input the tag vocabulary into the sequence generation model, wherein the sequence generation model assigns an initial default vector to each tag in the tag vocabulary.
[0066] (1.3) Input the training data into the sequence generation model, which converts the sentences in the training data into index vectors of keywords.
[0067] In this invention, pre-training the sequence generation model is equivalent to configuring the model accordingly, enabling the sequence generation model to quickly and accurately perform hierarchical label classification according to the rules of the hierarchical label classification task.
[0068] Step (2): Input the text data into the pre-trained sequence generation model for hierarchical label classification. That is, by fully considering the correlation between labels and the potential restriction of coarse-grained labels on fine-grained labels, hierarchical labels are generated by sequence generation, and the label information is linked together, so as to efficiently and accurately perform hierarchical label text classification. Furthermore, by using a masking method to control the generation of hierarchical labels, the uncontrollability in the hierarchical label generation task is solved.
[0069] Specifically, the sequence generation model performs hierarchical label classification on text data as follows:
[0070] (2.1) Multi-granularity text feature vectors are extracted using sequence generation.
[0071] In this embodiment of the invention, the process of extracting multi-granularity text feature vectors using sequence generation is as follows:
[0072] (2.11) Please refer to Figure 3 The encoder on the left takes text data as input and encodes it using a sequence generation method to obtain text feature vectors for keywords. This is achieved through multiple encoding layers (token embedding layer, segment embedding layer, and position embedding layer) to produce text feature vectors for each keyword. Each keyword's text feature vector is composed of three parts: a character vector, a segment vector, and a position vector. These text feature vectors are defined as follows:
[0073] e = e token +e segment +e position
[0074] Among them, e token This refers to the character vector encoded by the sequence generation model, representing character-level semantic information, e segment This refers to the segment vector encoded by the sequence generation model, used to distinguish sentence pairs, e position It refers to the position vector encoded by the sequence generation model, used to introduce position information.
[0075] (2.12) The text feature vectors of each input coding layer are summed to obtain the coding vectors of each coding layer corresponding to the keywords.
[0076] Specifically, the encoder of the sequence generation model consists of multiple stacked encoding layers (Layer_1, Layer_2 to Layer_n), each containing different semantic information. The final encoding vector is formed by summing the text feature vectors of each layer. The summation formula is as follows:
[0077]
[0078] Where L refers to the number of layers in the sequence generation model, h ik h represents the encoding vector of the text feature vector of the i-th keyword in the k-th layer of the sequence generation model. i This represents the final encoding vector generated by the model for the i-th sequence.
[0079] (2.2) An attention mechanism is used to encode the multi-granularity text feature vectors and the predicted upper-level labels to obtain the encoding vector containing the upper-level label information. That is, features are extracted from the multi-granularity information of the text data and fused. The attention mechanism is used to obtain the characteristic information of the keywords, which enriches the information contained in the encoding vector of the model and improves the classification effect of the model.
[0080] Specifically, the process for obtaining the encoding vector containing the label information of the previous level is as follows:
[0081] (2.21) Vectorize the previous level label to obtain the label vector. The previous level label is predicted by the sequence generation model and iteratively updated, that is, the label predicted in the previous step is converted into the corresponding vector representation g(y). t-1 When t=1, g(y0) is the initial default vector.
[0082] (2.22) The value vector V and key vector K are obtained by mapping the character-level encoding vector H, and the index vector q is obtained by mapping the label vector, as shown in the following formula:
[0083] K = W k HV = W v H q = Wq g(y t-1 )
[0084] Among them, W k W v and W q These are preset weight parameters, H = [h1, h2, ..., h m ], K = [k1,k2,...,k m V = [v1, v2, ..., v] m ].
[0085] (2.23) Perform cross-attention calculation based on the index vector q and the key vector K to obtain the attention weight vector a. t The formula is as follows:
[0086]
[0087] Here, d refers to the dimension of the index vector q.
[0088] (2.24) The Softmax function is used to adjust the attention weight vector a t Perform normalization calculations to obtain the normalized weight vector s t The formula is as follows:
[0089]
[0090] (2.25) Based on the normalized weight vector s t The value vector V is weighted and summed to obtain the encoded vector r containing the label information of the previous level. t The formula is as follows:
[0091] (2.3) A time-series network is used to decode the encoded vector containing the label information of the previous level to predict the label of the next level, and the label of the previous level is iteratively updated. Please refer to [link to relevant documentation]. Figure 3 Right-side decoder.
[0092] In this embodiment of the invention, the process of decoding the encoded vector containing the label information of the previous level using a time series network is as follows:
[0093] (2.31) The encoding vector r containing the label information of the previous level t The input is decoded by the LSTM model to obtain the intermediate vector l. t The formula is as follows:
[0094] l t =LSTM(r t )
[0095] (2.32) For the intermediate vector l t Perform a linear transformation to obtain the prediction vector o for the next level of labels. t The formula is as follows:
[0096] o t =f(W d l t )
[0097] Among them, W d is the weight parameter, f represents the non-linear activation function, and iterative updates are performed on the labels of the previous level until all labels are predicted.
[0098] (2.4) Use the mask operation (MASK) to control the generation of all level labels.
[0099] In this embodiment of the invention, the mask operation control process is as follows:
[0100] (2.41) Mask some of the hierarchical labels to obtain the indicator vector I. t The specific rules are as follows:
[0101]
[0102] Where t∈{1,2,…,N}, and N is the number of levels. This represents the label range of the t-th layer. This indicates that the range of label values in layer t is related to the range of values in layer t-1.
[0103] The following example uses a level 3 label to illustrate the indicator vector I. t The calculation.
[0104] For example: the hierarchy label is level 3:
[0105] If the current time step t = 1:
[0106]
[0107] Where Y j1 This indicates the range of values for the first-level tag. This indicates the number of first-level tags.
[0108] If the current time step t = 2, the first-level label is Y2, that is, j1 = 2:
[0109]
[0110] in, This indicates the range of values for the second-layer label when the first layer is Y2. This indicates the number of second-level tags, and the number of second-level tags is related to the number of first-level tags.
[0111] If the current time step t = 3, the first-level label is Y2, and the second-level label is Y. 2,1 :
[0112]
[0113] Among them, Y 2,1,j3 This indicates that the first layer is Y2 and the second layer is labeled Y. 2,1 At that time, the value range of the third-level label, This indicates the number of third-level tags, and the number of third-level tags is related to the number of second-level tags;
[0114] (2.42) Calculate the probability values of all labels based on the prediction vector and the indicator vector, and select labels based on their probability values, as shown in the following formula:
[0115] y t =softmax(o t +I t )
[0116] That is, when the probability value is greater than the threshold, it is predicted as the label of the next level.
[0117] Example 2
[0118] This invention provides a hierarchical label text classification system, which corresponds one-to-one with the method described in Embodiment 1. Please refer to [link / reference]. Figure 4 The system includes:
[0119] A sequence generation model building unit, wherein the sequence generation model building unit constructs a sequence generation model based on a hierarchical label classification task;
[0120] A sequence generation model pre-training unit, which is used to pre-train the sequence generation model;
[0121] A sequence generation model unit is used to input text data into a pre-trained sequence generation model and predict hierarchical labels on the text data.
[0122] The sequence generation model unit includes:
[0123] A text feature vector extraction module, wherein the text feature vector extraction module extracts multi-granular text feature vectors by using a sequence generation method;
[0124] The attention mechanism module uses an attention mechanism to encode multi-granularity text feature vectors and predicted upper-level labels to obtain an encoded vector containing upper-level label information.
[0125] The hierarchical label prediction module uses a time series network to decode the encoded vector containing the label information of the previous level in order to predict the label of the next level, and iteratively updates the label of the previous level.
[0126] A mask operation control module, which uses mask operations to control the generation of all levels of labels.
[0127] Example 3
[0128] This invention provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the computer program executes a hierarchical label text classification method as described in Embodiment 1 when the processor is running.
[0129] The computer device provided in this embodiment can implement the method described in Embodiment 1. To avoid repetition, it will not be described again here.
[0130] Example 4
[0131] This invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the hierarchical label text classification method described in Embodiment 1.
[0132] The computer-readable storage medium provided in this embodiment can implement the method described in Embodiment 1. To avoid repetition, it will not be described again here.
[0133] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0134] The memory can be used to store the computer program and / or modules. The processor implements various functions of the hierarchical label text classification system of the invention by running or executing the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart memory card, secure digital card, flash memory card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0135] If a hierarchical label text classification system is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program that can be stored in a computer-readable storage medium. When executed by a processor, this computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory, random access memory, dot carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.
[0136] The basic concepts of this invention have been described. It is obvious to those skilled in the art that the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.
[0137] Computer storage media may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and suitable combinations thereof. Computer storage media can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program code located on the computer storage medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.
Claims
1. A hierarchical label text classification method, characterized in that, The method includes the following steps: (1) Construct a sequence generation model based on the hierarchical label classification task, and pre-train the sequence generation model; The sequence generation task is defined as follows: ; Where x refers to text data containing m keywords, y refers to the set of n tags corresponding to x, y0 represents the top-level tag, y1 represents the first-level tag, y2 represents the second-level tag, and so on. Therefore, y... i-1 This represents the label at level i-1; (2) Input the text data into a pre-trained sequence generation model. The sequence generation model adopts a seq2seq structure, including an encoder and a decoder. The encoder encodes the text data based on the sequence generation method, and the decoder predicts the next level label from the top-level label through an event sequence network. The sequence generation model processes the text data as follows: (2.1) Extract multi-granularity text feature vectors using sequence generation; The process of extracting multi-granularity text feature vectors using sequence generation is as follows: (2.11) Input the text data into several encoding layers of the encoder. The encoding layers encode the text data based on sequence generation to obtain the text feature vector of the keywords. Each keyword's text feature vector is composed of three parts: a character vector, a segment vector, and a position vector. The text feature vector is defined as follows: ; in, It refers to the character vector encoded by the sequence generation model, representing character-level semantic information. This refers to the segment vector encoded by the sequence generation model, used to distinguish sentence pairs. It refers to the position vector encoded by the sequence generation model, used to incorporate positional information; (2.12) Sum the text feature vectors of each coding layer to obtain the coding vectors of each coding layer corresponding to the keywords; The encoder of the sequence generation model consists of multiple stacked encoding layers, each containing different semantic information. The final encoding vector is formed by summing the text feature vectors of each layer. The summation formula is as follows: ; Where L refers to the number of layers in the sequence generation model. This represents the encoding vector of the text feature vector of the i-th keyword in the k-th layer of the sequence generation model. This represents the final encoding vector generated by the model for the i-th sequence; (2.2) An attention mechanism is used to encode the multi-granularity text feature vectors and the predicted upper-level labels to obtain an encoded vector containing upper-level label information; The process of using an attention mechanism to encode multi-granularity text feature vectors and predicted parent-level labels to obtain encoded vectors containing parent-level label information is as follows: (2.21) Vectorize the upper-level label to obtain the label vector, wherein the upper-level label is predicted by the sequence generation model and iteratively updated; (2.22) Value vectors and key vectors are obtained by mapping character-level encoded vectors, and corresponding index vectors are obtained by mapping label vectors; (2.23) Perform cross-attention calculation based on the index vector and key vector to obtain the attention weight vector; (2.24) Normalize the attention weight vector to obtain the normalized weight vector; (2.25) The value vector is weighted and summed based on the normalized weight vector to obtain the encoded vector containing the label information of the previous level; (2.3) A time series network is used to decode the encoded vector containing the label information of the previous level in order to predict the label vector of the next level, and the label of the previous level is iteratively updated; The process of using a time series network to decode the encoded vector containing the label information of the previous level in order to predict the label of the next level, and iteratively updating the label of the previous level, is as follows: (2.31) Input the encoded vector containing the label information of the previous level into the decoder for decoding to obtain the intermediate vector, as shown in the following formula: ; in, Represents the intermediate vector. This represents an encoded vector containing information from the previous level's label. Indicates decoder; (2.32) Perform a linear transformation on the intermediate vector to obtain the predicted vector of the next level label, and iteratively update the previous level label until all label predictions are completed, as shown in the following formula: ; in, This represents the predicted vector for the next level of labels. Here are the weight parameters, and f represents the activation function; (2.4) Use masking operations to control the generation of labels at all levels; The process of controlling the generation of all levels of tags using mask operations is as follows: (2.41) Mask a portion of the predicted vectors to obtain the indicator vectors; (2.42) Calculate the probability values of all labels based on the prediction vector and the indicator vector, and select labels based on their probability values. When the probability value is greater than the threshold, the label is predicted as the next level label. The formula is as follows: ; in, This represents the probability value of the label.
2. The hierarchical label text classification method according to claim 1, characterized in that: The process for pre-training a sequence generation model is as follows: (1.1) Obtain training data and construct a label vocabulary based on the training data; (1.2) Input the tag vocabulary into the sequence generation model, wherein the sequence generation model assigns an initial default vector to each tag in the tag vocabulary; (1.3) Input the training data into the sequence generation model, which converts the sentences in the training data into index vectors of keywords.
3. A hierarchical label text classification system, characterized in that, The system refers to claim 1, and the system comprises: A sequence generation model building unit, wherein the sequence generation model building unit constructs a sequence generation model based on a hierarchical label classification task; A sequence generation model pre-training unit, which is used to pre-train the sequence generation model; A sequence generation model unit is used to input text data into a pre-trained sequence generation model and predict hierarchical labels on the text data. The sequence generation model unit includes: A text feature vector extraction module, wherein the text feature vector extraction module extracts multi-granular text feature vectors by using a sequence generation method; The attention mechanism module uses an attention mechanism to encode multi-granularity text feature vectors and predicted upper-level labels to obtain an encoded vector containing upper-level label information. The hierarchical label prediction module uses a time series network to decode the encoded vector containing the label information of the previous level in order to predict the label of the next level, and iteratively updates the label of the previous level. A mask operation control module, which uses mask operations to control the generation of all levels of labels.
4. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements a hierarchical label text classification method as described in any one of claims 1-2.
5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the hierarchical label text classification method according to any one of claims 1-2.