A multi-label text classification method, device, chip and terminal
Patent Information
- Application Number
- CN202410767310.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-14
- Publication Date
- 2026-08-18
- Estimated Expiration
- 2044-06-14
AI Technical Summary
[0004]针对现有技术中所存在的不足，本发明提供了一种多标签文本分类方法、装置、芯片及终端，其解决了现有技术中多标签文本分类方法没有充分利用原始文本的内容，难以大幅提高分类准确性的问题
[0033]The multi-label text classification method provided in this invention obtains a text representation with label semantics, i.e., second text information, through multi-head attention calculation and cross-attention calculation; it obtains text with label joint text embedding, i.e., label information, by combining text and labels through label-related attention weights; and it performs the classification task after fusing the two to obtain fused information. This fully utilizes the semantic information of the original text. Labels are no longer just used by the classifier to interpret text information, but are used to provide an effective intermediate representation of the semantic overlap in the original text based on the second text information and label information, i.e., the aforementioned fused information. This allows the connection between text and labels to participate more directly in the text classification task, comprehensively improving the accuracy of multi-label text classification.
Smart Images
Figure CN118733769B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text classification technology, and in particular to a multi-label text classification method, apparatus, chip, and terminal. Background Technology
[0002] In the field of deep learning, multi-label text classification methods improve the generalization ability and adaptability of deep learning models in different domains or tasks and increase classification accuracy by optimizing the classification performance of deep learning models, optimizing mechanisms to capture the correlation information between labels, improving loss functions and evaluation metrics, and utilizing techniques such as transfer learning and adaptive learning.
[0003] However, the above methods do not make full use of the content of the original text, making it difficult to significantly improve classification accuracy. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a multi-label text classification method, device, chip, and terminal, which solves the problem that existing multi-label text classification methods do not fully utilize the content of the original text and are difficult to significantly improve classification accuracy.
[0005] According to embodiments of the present invention, a first aspect provides a multi-label text classification method, comprising:
[0006] The original text is vectorized and the word vector dimensions of the first tag vector set are reset to obtain the text vector set and the second tag vector set;
[0007] Wherein, the first tag vector set is generated based on the original text, and the second tag vector is a first tag vector with the word vector dimensions reset;
[0008] The text vector set is individually encoded using multi-head attention computation to obtain first text information; the first text information is then jointly encoded using cross-attention computation in conjunction with the second tag vector set to obtain second text information.
[0009] Based on the set of text vectors, extract the tag-related attention weights of each second tag vector, and represent the set of second tag vectors as tag information according to the tag-related attention weights of each second tag vector;
[0010] The fusion information is obtained based on the second text information and the tag information;
[0011] The original text is classified based on the fused information.
[0012] Optionally, before obtaining the first text information by individually encoding the text vector set through multi-head attention computation, the process includes:
[0013] The deformed dimension is calculated by combining the word vector dimension of the text vector set with the number of attention heads used in the multi-head attention computation; the deformed dimension is the dimension of the multi-head attention computation output matrix.
[0014] Optionally, by combining the second set of label vectors and performing joint encoding on the first text information through cross-attention computation, the second text information is obtained, including:
[0015] Using the second set of label vectors as input to calculate the first cross-attention matrix, and using the first text information as input to calculate the second and third cross-attention matrices, the outputs of the first, second, and third cross-attention matrices are obtained.
[0016] The second text information is calculated based on the outputs of the first cross-attention matrix, the second cross-attention matrix, and the third cross-attention matrix.
[0017] Optionally, after extracting the label-related attention weights for each second label vector based on the text vector set, the second label vector is represented as label information according to the label-related attention weights, including:
[0018] The relative spatial information between consecutive text vectors in the text vector set is captured by the local matrix of the tag-related attention weights, and the relationship between each text vector in the text vector set and all second vector tags in the second tag vector set is obtained to represent the tag information.
[0019] Optionally, the label information includes:
[0020] The relationship between each text vector in the text vector set and all second vector labels in the second label vector set is normalized and enhanced by the label-related attention weights, and the output weight matrix represents the label information.
[0021] Optionally, the fusion information is obtained based on the second text information and the tag information, including:
[0022] The second text information and the tag information are sampled to the maximum value respectively, and the maximum sampling results are concatenated to obtain the fused information.
[0023] Optionally, the original text is classified according to the fusion information, including:
[0024] The fused information is mapped to each first label vector in the first label vector set through a fully connected layer, and the probability of each first label vector is calculated using a sigmoid activation function.
[0025] The second aspect provides a multi-label text classification device, including:
[0026] The word embedding module is used to vectorize the original text and reset the word vector dimension of the first tag vector set to obtain a text vector set and a second tag vector set; wherein, the first tag vector set is generated based on the original text, and the second tag vector is the first tag vector with the word vector dimension reset;
[0027] The encoding module is used to encode the text vector set individually through multi-head attention computation to obtain first text information; it is also used to combine the second label vector set and jointly encode the first text information through cross-attention computation to obtain second text information.
[0028] The tag embedding module is used to extract the tag-related attention weight of each second tag vector based on the text vector set, and represent the second tag vector set as tag information according to the tag-related attention weight of each second tag vector;
[0029] The fusion module is used to obtain fused information based on the second text information and the tag information;
[0030] The text classification module is used to classify the original text based on the fusion information.
[0031] Thirdly, a chip is provided, including a first processor for calling and running a computer program from a first memory, causing a device equipped with the chip to perform the various steps of the multi-label text classification method described above.
[0032] Fourthly, a terminal is provided, including a second memory, a second processor, and a computer program stored in the second memory and executable on the second processor. When the second processor executes the computer program, it implements the various steps of the multi-label text classification method described above.
[0033] The multi-label text classification method provided in this invention obtains a text representation with label semantics, i.e., second text information, through multi-head attention calculation and cross-attention calculation; it obtains text with label joint text embedding, i.e., label information, by combining text and labels through label-related attention weights; and it performs the classification task after fusing the two to obtain fused information. This fully utilizes the semantic information of the original text. Labels are no longer just used by the classifier to interpret text information, but are used to provide an effective intermediate representation of the semantic overlap in the original text based on the second text information and label information, i.e., the aforementioned fused information. This allows the connection between text and labels to participate more directly in the text classification task, comprehensively improving the accuracy of multi-label text classification. Attached Figure Description
[0034] Figure 1 This is a flowchart illustrating the implementation of the multi-label text classification method according to an embodiment of the present invention.
[0035] Figure 2 This is a schematic diagram of experimental data for the multi-label text classification method in an embodiment of the present invention;
[0036] Figure 3 This is a schematic diagram of the composition structure of the multi-label text classification device according to an embodiment of the present invention;
[0037] Figure 4 This is a schematic diagram of the calculation process of the multi-label text classification method according to an embodiment of the present invention;
[0038] Figure 5 This is a basic structural block diagram of a terminal according to an embodiment of the present invention. Detailed Implementation
[0039] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0040] Example 1
[0041] In practical applications of multi-label text classification methods, the original text can be assigned to multiple category labels. This invention, however, performs multi-label text classification based on the original text and a second label vector set. Based on this, as... Figure 1 As shown in the embodiment of the present invention, a multi-label text classification method includes:
[0042] S1. Vectorize the original text and reset the word vector dimension of the first tag vector set to obtain the text vector set and the second tag vector set.
[0043] It is understandable that the first tag vector set represents multiple first tag vectors, the second tag vector set represents multiple second tag vectors, and the text vector set represents multiple text vectors. It should be noted that the first tag vector set is generated based on the original text, and the second tag vector is a first tag vector with its word vector dimensions reset.
[0044] In one embodiment, one way to vectorize the original text in step S1 above includes: using a word vector embedding model pre-trained with a 300-dimensional GloVe (Global Vectors) to vectorize the original text. The resulting multiple text vectors, i.e., the text vector model, have good representational meaning and can initially represent the semantics of the original text.
[0045] In step S1 above, the word vector dimension of the first tag vector set is reset so that the word vector dimension of the text vector set is consistent with the word vector dimension of the second tag vector, which facilitates calculation.
[0046] A better approach involves simultaneously using a word embedding model to vectorize the original text and re-dimensioning the word vectors of the second label vectors. Furthermore, the word embedding model uses positional encoding to store the relative or absolute positions of each text vector in the text vector set.
[0047] For example, the following formula is introduced to explain in detail the implementation of the above steps:
[0048] The set of text vectors is represented as W = {w1, w2, ..., w...} n The second set of label vectors can be represented as L = {l1, l2, ..., l}. l}, W={w1,w2,…,w n}L={l1,l2,…,l l Where n represents the length of the original text, l represents the number of second label vectors in the second label vector set, and w i Let l represent any text vector in the set of text vectors. i It represents any second label vector in the set of second label vectors.
[0049] In this embodiment of the invention, the input of the word vector embedding model references the input of the Transformer. Therefore, when the word vector embedding model also uses positional encoding to store the relative or absolute positions of each text vector in the text vector set, this positional encoding is represented by PE. The set of text vectors can be represented as: H = {w1 + PE1, w2 + PE2, ..., w} n +PE n}
[0050] S2. Encode the text vector set individually using multi-head attention computation to obtain first text information; combine the second label vector set and jointly encode the first text information using cross-attention computation to obtain second text information.
[0051] In step S2 above, the separate encoding calculated by multi-head attention is used to obtain the first text information that integrates global information and local context. The first text information is more accurate in representing the semantics of the original text than the set of text vectors.
[0052] Furthermore, before individually encoding the text vector set using multi-head attention computation, this embodiment of the invention makes adaptive adjustments to the multi-head attention computation to make the output more focused on semantic information. Therefore, before performing step S2 above, the following steps are also included:
[0053] The deformed dimension is calculated by combining the word vector dimension of the text vector set with the number of attention heads used in the multi-head attention computation; the deformed dimension is the dimension of the multi-head attention computation output matrix.
[0054] For example, the following formula is introduced to explain in detail the implementation of the above steps:
[0055] The formula for calculating the deformation dimension is as follows: Where C is the transformation dimension, d is the word vector dimension of the text vector set, and h is the number of attention heads used in the multi-head attention computation.
[0056] Based on the transformed multi-head attention calculation matrix Q, K, V, the output matrix of the multi-head attention calculation has a dimension of C. In the multi-head attention calculation, the attention score is calculated for each attention head, using the following formula:
[0057] head i =Attention(Q) i ,K i V i )
[0058] Multihead(H)=Concat(head1,head2,…,head h W o
[0059] Meanwhile, multi-head attention computation uses the FeedFoward structure. The process of encoding the text vector set separately through multi-head attention computation can be represented as follows:
[0060] =layerNorm(H+Multihead(H))
[0061] Hf = max(0, HW1+b1)W2+b2
[0062] H = layerNorm(H + Hf)
[0063] In traditional convolutional networks, the receptive field typically increases with the number of layers. This invention, by introducing an attention mechanism, adjusts the size and stride of the convolutional kernel based on attention information, thereby changing the size of the receptive field. It integrates local information and local context to output the first text information. Therefore, the output first text information can be represented as:
[0064]
[0065] Where H = {h1, h2, ..., h n} represents the first text information, i.e., the multi-head self-attention computation pairs the text vector set H = {w1 + PE1, w2 + P2, ... w}. n +PE n The output obtained by encoding} separately, c j This represents the output of the j-th layer of the attention convolution.
[0066] In step S2 above, the joint cross-attention calculation and the joint encoding of the second label vector set are used to output second text information with label semantics. Therefore, compared to the first text information, the second text information represents not only the semantics of the original text but also the semantics from the labels. In one embodiment, one implementation of the joint encoding is as follows:
[0067] Using the second set of label vectors as input to calculate the first cross-attention matrix, and using the first text information as input to calculate the second and third cross-attention matrices, the outputs of the first, second, and third cross-attention matrices are obtained.
[0068] The second text information is calculated based on the outputs of the first cross-attention matrix, the second cross-attention matrix, and the third cross-attention matrix.
[0069] For example, the following formula is introduced to explain in detail the implementation of the above steps:
[0070] Q′=LW1;K′=M w W2; V′=M w W3;
[0071] Where W1, W2, and W3 are parameter matrices, Q′ represents the first cross-attention matrix, K′ represents the second cross-attention matrix, V′ represents the third cross-attention matrix, L is the second set of label vectors, and M... w This is the first text information.
[0072] Based on the outputs of the first, second, and third cross-attention matrices, the second text information is calculated using the following formula:
[0073]
[0074] Where Y represents the second text information, and d is the word vector dimension of the text vector set. The above formula means that by calculating the similarity between the label and the text, and then multiplying the output obtained by softmax normalization of the similarity with matrix V, we obtain the final output, which is the text representation with label semantics, i.e., the second text information.
[0075] S3. Based on the text vector set, extract the tag-related attention weights of each second tag vector, and represent the second tag vector set as tag information according to the tag-related attention weights of each second tag vector.
[0076] In step S3 above, the label-related attention weight represents the influence of each text vector in the text vector set on each second label vector in the second label vector set. For example, for second label vector A, the influence of text vectors B1 to B3 on second label vector A is obtained. Based on the label-related attention weight of each second label vector, the second label vector set is represented as label information. Therefore, the label information includes not only the second label vectors but also information embedded in the joint text vectors of the second label vectors. In other words, the label information is the text of the joint text embedding of labels, which includes semantic content from the text vectors.
[0077] For example, the following formula is introduced to explain in detail the implementation of the above steps:
[0078] G s =LH T
[0079] G s For the label-related attention weights, L represents the second label set, and H = {h1, h2, ..., h...} n} represents the first text information, i.e., the multi-head self-attention computation on the text vector set H = {w1 + PE1, w2 + PE2, ... w}. n +PE n The output is obtained by encoding it separately.
[0080] Since the text vector set comes from the original text, and the words in the original text are continuous, this embodiment of the invention considers the continuity of each text vector in the text vector set when representing label information, so that the label information can reflect a higher level of spatial information between each word in the original text, thereby expanding the connection between each text vector in the text vector set and all second vector labels in the second label vector set. Therefore, in step S3 above, after extracting the label-related attention weight of each second label vector based on the text vector set, the second label vector is represented as label information according to the label-related attention weight, including:
[0081] The relative spatial information between consecutive text vectors in the text vector set is captured by the local matrix of the tag-related attention weights, and the relationship between each text vector in the text vector set and all second vector tags in the second tag vector set is obtained to represent the tag information.
[0082] In practical applications, using a local matrix of tag-related attention weights is an efficient way to introduce continuity among the various text vectors. For example, the following formula is used to explain the implementation of the above steps in detail:
[0083] The relative spatial information between consecutive text vectors in the text vector set is captured by the local matrix of the label-related attention weights, as shown in the formula:
[0084] M S =Relu(G s-r,s+r W+b);
[0085] In the above formula, the relative spatial length is defined as r, then sr,s+r is a length module with a length of 2r+1 centered at s.
[0086] Assuming the label-related attention weight is given, for the second label vector A, to obtain the influence of text vectors B1 to B3 on the second label vector A, the above formula can capture the relationship between text vectors B1, B2, and B3 when text vectors B1 to B3 are consecutive text vectors.
[0087] In one embodiment, the label information includes:
[0088] The relationship between each word in the text vector set and all labels in the second label vector is normalized, and the label-related attention weights are enhanced to output a weight matrix representing the label information.
[0089] For example, the following formula is introduced to explain in detail the implementation of the above steps:
[0090] Ml =softmax(M s );
[0091] M w =M l H T ;
[0092] Where Ms represents the relative spatial information between consecutive text vectors in the text vector set, M l For normalized M s M w For the enhanced M l H is a set of text vectors.
[0093] The tag information will ultimately be in M w express.
[0094] S4. Obtain fusion information based on the second text information and the tag information.
[0095] One way to implement step S4 above is as follows:
[0096] The second text information and the tag information are sampled to the maximum value respectively, and the maximum sampling results are concatenated to obtain the fused information.
[0097] The above steps can be described by the following formula:
[0098] M k =mapool(M w );
[0099] Y k =maxpool(Y);
[0100] M o =Concat(M k ,Y k );
[0101] The fusion of the second text information and the tag information is a fusion of text representation with tag semantics and text embedded with tag joint text, so that the fused information is a globally tag-related information representation.
[0102] S5. Classify the original text according to the fusion information.
[0103] One way to implement step S5 above is as follows:
[0104] The fused information is mapped to each first label vector in the first label vector set through a fully connected layer, and the probability of each first label vector is calculated using a sigmoid activation function.
[0105] The above steps can be described by the following formula:
[0106] y = sigmoid(WM O +b)
[0107] Steps S1 to S5 above involve obtaining a text representation with label semantics, i.e., the second text information, through multi-head attention calculation and cross-attention calculation; and obtaining the text with label joint text embedding, i.e., label information, by combining the text and labels through label-related attention weights. The fused information is then used for classification tasks, fully utilizing the semantic information of the original text. Labels are no longer merely used by the classifier to interpret text information, but rather serve as an effective intermediate representation of semantic overlap in the original text based on the second text information and label information—the aforementioned fused information. This allows the connection between text and labels to participate more directly in the text classification task, comprehensively improving the accuracy of multi-label text classification. Furthermore, this embodiment of the invention utilizes a local matrix to expand the connection between each text vector in the text vector set and all second vector labels in the second label vector set, in order to model the relevance between text and words.
[0108] Example 2
[0109] This invention also provides experimental data on the application of the multi-label text classification task performed according to steps S1 to S5 above on different datasets. For example... Figure 2 As shown, the experimental results are compared with other benchmark models on the RCV1-V2 dataset. The four indicators, Hamming loss (HL), micro-F1 score, micro-precision, and micro-recall, are 0.0073, 0.880, 0.906, and 0.856, respectively. This demonstrates that the multi-label text classification method provided in this embodiment of the invention has a comprehensive improvement in classification performance and achieves better classification results.
[0110] Example 3
[0111] To address the aforementioned technical problems, embodiments of the present invention also provide a multi-label text classification device. Please refer to [link / reference] for details. Figure 3 , Figure 3 The basic structural block diagram of the multi-label text classification device 30 of this embodiment includes:
[0112] The word embedding module 31 is used to vectorize the original text and reset the word vector dimension of the first tag vector set to obtain a text vector set and a second tag vector set; wherein, the first tag vector set is generated based on the original text, and the second tag vector is the first tag vector with the word vector dimension reset.
[0113] Encoding module 32 is used to encode the text vector set individually through multi-head attention computation to obtain first text information; it is also used to combine the second label vector set and jointly encode the first text information through cross-attention computation to obtain second text information.
[0114] The tag embedding module 33 is used to extract the tag-related attention weight of each second tag vector based on the text vector set, and represent the second tag vector set as tag information according to the tag-related attention weight of each second tag vector;
[0115] The fusion module 34 is used to obtain fusion information based on the second text information and the tag information;
[0116] The text classification module 35 is used to classify the original text based on the fusion information.
[0117] like Figure 4 As shown in the diagram, this embodiment of the invention provides a schematic diagram of the calculation process to illustrate the above-described multi-label text classification method and multi-label text classification device. Figure 4 In the above steps S101 and the word embedding module are implemented as follows: Figure 4 D1, the above steps S102 and the implementation of the encoding module are as follows Figure 4 D2, the above steps S103 and the tag embedding module are implemented as follows Figure 4 D3, the above steps S104 and the implementation of the fusion module are as follows Figure 4 In this context, D2 and D3 refer to the process of obtaining fused information through D4. For example, Figure 4 In this embodiment, D1 can represent a 300-dimensional GloVe pre-trained word vector embedding module, D2 can represent an encoding module and a cross-attention module based on a multi-head attention mechanism, D3 can represent a label attention model that extracts text attention from labels, and the embodiments of the present invention do not limit D4 and the network model used for text classification tasks based on D4.
[0118] Example 4
[0119] To address the aforementioned technical problems, embodiments of the present invention also provide a chip, which can be a general-purpose processor or a dedicated processor. The chip includes a processor configured to support a terminal in executing the aforementioned steps, such as retrieving and running a computer program from memory, causing a device equipped with the chip to execute the multi-label text classification method described in the various embodiments above.
[0120] Optionally, in some examples, the chip may also include a transceiver for receiving control from the processor to support the terminal in performing the aforementioned steps to implement the multi-label text classification method in the various embodiments described above.
[0121] Optionally, the chip may also include a storage medium.
[0122] It should be noted that the chip can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits capable of performing the various functions described throughout this application.
[0123] Example 5
[0124] The present invention also provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multi-label text classification method as described in the above embodiments.
[0125] Please refer to the details. Figure 5 , Figure 5 The diagram illustrates a basic structural block diagram of a terminal, which includes a processor, a non-volatile storage medium, a memory, and a network interface connected via a system bus. The non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database may store a sequence of control information. When executed by the processor, the computer-readable instructions enable the processor to implement a multi-label text classification method. The processor provides computational and control capabilities to support the operation of the entire terminal. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to perform a multi-label text classification method. The network interface is used for communication with the terminal. Those skilled in the art will understand that the structure shown in the diagram is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. A specific terminal may include more or fewer components than shown in the diagram, or combine certain components, or have different component arrangements.
[0126] Those skilled in the art will understand that the terms "terminal" and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, electronic devices having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such electronic devices may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptop and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "terminal" or "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Terminal" or "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.
[0127] The present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the multi-label text classification method described in the above embodiments.
[0128] This embodiment also provides a computer program that can be distributed on a computer-readable medium and executed by a computing device to implement at least one step of the multi-label text classification method described in the above embodiments; and in some cases, at least one step shown or described may be executed in a different order than that described in the above embodiments.
[0129] This embodiment also provides a computer program product, including a computer-readable device on which the computer program as shown above is stored. In this embodiment, the computer-readable device may include the computer-readable storage medium as shown above.
[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).
[0131] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0132] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0133] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. A multi-label text classification method, characterized in that, include: The original text is vectorized and the word vector dimensions of the first tag vector set are reset to obtain the text vector set and the second tag vector set; Wherein, the first tag vector set is generated based on the original text, and the second tag vector is a first tag vector with the word vector dimensions reset; The text vector set is individually encoded using multi-head attention computation to obtain first text information; the first text information is then jointly encoded using cross-attention computation in conjunction with the second tag vector set to obtain second text information. Based on the aforementioned set of text vectors, extract the label-related attention weights for each second label vector; The relative spatial information between consecutive text vectors in the text vector set is captured by the local matrix of the label-related attention weights. The calculation formula is: In the formula, Relative spatial length For The length of the center is The length module, This is a local matrix of label-related attention weights. This is the weight matrix. For bias terms; Obtain the relationship between each text vector in the text vector set and all second tag vectors in the second tag vector set to represent the tag information; The fusion information is obtained based on the second text information and the tag information; The original text is classified based on the fused information.
2. The multi-label text classification method as described in claim 1, characterized in that, Before obtaining the first text information by individually encoding the text vector set through multi-head attention computation, the process includes: The deformed dimension is calculated by combining the word vector dimension of the text vector set with the number of attention heads used in the multi-head attention computation; the deformed dimension is the dimension of the multi-head attention computation output matrix.
3. The multi-label text classification method as described in claim 1, characterized in that, By combining the second set of label vectors and performing joint encoding on the first text information through cross-attention computation, the second text information is obtained, including: Using the second set of label vectors as input to calculate the first cross-attention matrix, and using the first text information as input to calculate the second and third cross-attention matrices, the outputs of the first, second, and third cross-attention matrices are obtained. The second text information is calculated based on the outputs of the first cross-attention matrix, the second cross-attention matrix, and the third cross-attention matrix.
4. The multi-label text classification method as described in claim 1, characterized in that, The label information includes: The relationship between each text vector in the text vector set and all second label vectors in the second label vector set is normalized and enhanced by the label-related attention weights, and the output weight matrix represents the label information.
5. The multi-label text classification method as described in claim 1, characterized in that, Based on the second text information and the tag information, fused information is obtained, including: The second text information and the tag information are sampled to the maximum value respectively, and the maximum sampling results are concatenated to obtain the fused information.
6. The multi-label text classification method as described in claim 1, characterized in that, Based on the fusion information, the original text is classified, including: The fused information is mapped to each first label vector in the first label vector set through a fully connected layer, and the probability of each first label vector is calculated using a sigmoid activation function.
7. A multi-label text classification device for implementing the multi-label text classification method according to any one of claims 1 to 6, characterized in that, include: The word embedding module is used to vectorize the original text and reset the word vector dimension of the first tag vector set to obtain a text vector set and a second tag vector set; wherein, the first tag vector set is generated based on the original text, and the second tag vector is the first tag vector with the word vector dimension reset; The encoding module is used to encode the text vector set individually through multi-head attention computation to obtain first text information; and to jointly encode the first text information by combining the second label vector set through cross-attention computation to obtain second text information. The tag embedding module is used to extract tag-related attention weights for each second tag vector based on the text vector set, and to capture the relative spatial information between consecutive text vectors in the text vector set through the local matrix of the tag-related attention weights. The calculation formula is: In the formula, Relative spatial length For The length of the center is The length module, This is a local matrix of label-related attention weights. This is the weight matrix. For bias terms; Obtain the relationship between each text vector in the text vector set and all second tag vectors in the second tag vector set to represent the tag information; The fusion module is used to obtain fused information based on the second text information and the tag information; The text classification module is used to classify the original text based on the fusion information.
8. A chip, characterized in that, include: A first processor is configured to retrieve and run a computer program from a first memory, causing a device equipped with the chip to perform the steps of the multi-label text classification method as described in any one of claims 1 to 6.
9. A terminal, characterized in that, The method includes a second memory, a second processor, and a computer program stored in the second memory and executable on the second processor, characterized in that the second processor executes the computer program to implement the steps of the multi-label text classification method as described in any one of claims 1 to 6.